Author|Walter Trojan
Translator | Lens
This article will discuss how artificial intelligence works in deep learning. If you want to study AI seriously, Linux and Python are essential tools. Only by choosing the right development tools can you get twice the result with half the effort. I have studied some related books and tutorials, and conducted a lot of programming tests. At present, I still have a lot of confusion, but at least I can give you an overview, such as what structures and methods are used in specialized intelligent neural networks (NN).
01
Neural network structure
A neural network (NN) consists of several layers, and each layer has several nodes. In Figure 1, the nodes in the same layer are arranged vertically. In theory, there can be any number of layers in the network, and each layer can have any number of nodes. The architecture is determined by the corresponding task, and the size depends on the resources owned by the computer platform. For example, if a neural network (NN) wants to classify the received objects according to pictures, the first layer will take over the data to complete the task and provide a corresponding number of input nodes for collection. If the low-resolution image is 28×28 pixels, then 784 nodes are needed to display the corresponding gray value. To obtain a color image, the number of nodes will be increased three times. During image acquisition, each input node will receive the gray value of the corresponding pixel.
Figure 1 Neural network structure
The results of image analysis show that each output result has a separate node in the output layer. If the neural network (NN) is to recognize 1000 objects, the layer needs to have the same number of output nodes. Each node holds a probability between 0 and 1.0 to indicate the reliability of the result. For example, when sampling a house cat image, the value of the associated node is 0.85, the value of the "tiger" node is 0.1, and the probability of other nodes is lower, so the result will be obvious.
During deep learning, the actual analysis work is done by the hidden layer. Depending on the task, any number of hidden layers can be used, and there are generally 100 to 200 hidden layers in use. In addition to the architecture shown above, there are more complex structures such as feedback or intermediate filters to improve accuracy. The number of nodes in each hidden layer is also arbitrary. In the above example for object detection, 3 hidden layers with 200 nodes can already provide satisfactory results.
If the human brain connects neurons through synapses to realize intelligence, then the nodes in the neural network (NN) are also connected to each other. Each node is connected to all nodes in subsequent layers. Each connection (shown by the arrow in Figure 2) contains a weight, and the weights between layers are stored in a matrix. Therefore, in the field of deep learning, choosing a programming language that is good at matrix operations can complete tasks easily and quickly.
Figure 2 Calculation within the node (https://commons.wikimedia.org/wiki/Artificial_neural_network)
How does a neural network (NN) achieve the desired result? Each node receives input signals from all nodes in the previous layer, which is labeled x in Figure 2. These values are multiplied by the weight value w and added together to get the net input value:
net=x1*w1+x2*w2+x3*w3…
The net input value is multiplied by the activation function and then sent to the output (to all nodes in the next layer). At the same time, the threshold of the node also determines whether the node is "triggered." The activation function shown in Figure 3 is assigned to each layer to ensure that the output value remains within the required range. For example, the Relu function suppresses all negative values and limits Sigmoid to between 0 and 1. These precautions can ensure that the NN operates within a clear range of values and will not "run out of control".
Figure 3 Commonly used activation functions
02
training
After recording the input data, the neural network (NN) iteratively performs a large number of calculations and outputs the results. This process is called "inference". How to get the expected intelligence? This is like teaching a child, which needs to be completed through education and training. For an untrained neural network (NN), the weight is usually a random number from -1 to +1, so only random results can be obtained. Therefore, training is required to feed data and known target output values to the network. After calculating with these values, the result is compared with the set value, and the difference is recorded through the loss function. These learning processes are also called "back propagation". Starting from the last layer, gradually adjust the weights to minimize losses. After training multiple times (usually millions of times) with different input data, all tasks are matched with appropriate weights, and the neural network (NN) can also analyze new untrained data.
Convolutional neural network is an architecture especially suitable for image and audio processing, and Maixduino can also support it. In a convolutional neural network, neurons (at least in some layers) are arranged in two dimensions, suitable for two-dimensional input data (such as images). Compared with the network shown in Figure 1 (where the activity of neurons depends on all neurons in the previous layer (by different weighting factors)), in the case of convolutional neural networks, the dependencies are simplified and local restrictions are imposed. . The activity of a neuron depends only on the values of 3×3 (for example) neurons, which are located in the layer before the neuron and have the same weighting factor. Using such a network can well detect small-scale structures such as straight lines, curves, dots and other patterns. In subsequent layers, more complex details can be detected and eventually the entire face can be detected.
As far as programming is concerned, a neural network (NN) is similar to a spreadsheet, which predefines the cell arrangement of calculation instructions, and uses weights to access a multi-dimensional matrix during execution. When classifying new input data, it will go through all layers from input to output, and output the corresponding probability of the result on the corresponding node. A well-trained neural network (NN) can reach a value around 0.9. During the training process, the interference will propagate back and re-adjust the weights from the input. It looks complicated at first glance, and there are a lot of tools and libraries that can be used in the implementation process.
03
Linux given Python
When dealing with AI issues, you must be prepared for learning. The easiest way is to use the Linux platform, because most of the tools on Linux are free and of high quality. In addition, Linux provides the same experience as Windows, but the software is packaged in a different way (usually better). I prefer Ubuntu, it has an LTS version (long-term support), at least 4 years of technical support. Linux also has other distribution versions (such as Debian, Mint, etc.), which can be selected according to personal preference. Linux can be installed in a virtual machine other than Windows, so no additional computer is required.
Why choose Python? Some people will say that Python is an interpreted programming language that runs very slowly. But this disadvantage is offset by many advantages: Python code eliminates parentheses and semicolons, and builds code blocks through indentation; has powerful data structures, such as lists, tuples, sets, and dictionaries; Python fully integrates matrix calculations Features, with a large number of highly modular AI frameworks and libraries, which can be fully or partially integrated (the modules are mainly written in C++, so they have excellent performance); through simple documentation, they can be easily installed and used. Choose your favorite Linux distribution, and install pip3 and Python 3!
04
Maixduino runs MicroPython
On systems with less memory, you can choose the lightweight Python version of MicroPython. It can be installed on platforms such as Maixduino and ESP32. MicroPython contains a basic command library and 55 other modules, which can be used for mathematics and system development. If you want to add a new version or AI model, you need to use the Kflash tool to refresh the program.
Install Kflash under Linux:
① Download the 1.5.3 or higher version of kflash_gui_v1.5.3_linux.tar.xz from the website link [2];
② Unzip to the specified folder tar xvf kflash_gui_ v1.5.3_linux.tar.xz;
③ Enter the folder /kflash_gui_v1.5.2_linux/kflash_gui;
④ Start /kflash_gui.
The graphical interface for starting Kflash is shown in Figure 4, and the firmware or AI model can be loaded into Maixduino.
Figure 4 Kflash
05
Maixduino install firmware
Although Maixduino is already equipped with MicroPython when it is delivered, it is still recommended to download the latest version. When writing this part of the code, the corresponding firmware name is v0.5.0, which can be downloaded through the website link [3]. Select maixpy_v0.5.0_8_g9c3b97f or higher, and then select maixpy_v0.5.0_8_g9c3b97f_minimum_with_ide_support.bin or higher on the next page. This file is about 700 KB and includes support for MaixPyIDE.
Use Kflash to quickly install new firmware. Click the "Open File" button to select the file, and then set the development board, port, baud rate and speed mode, as shown in Figure 5. Click "Download" to see the loading progress bar. After completion, you can execute Python commands (such as Putty) with a terminal emulator in Maixduino through the port /dev/ttyUSB0. Here is a set of commands:
>>> # Python Prompt
>>> import array as arr # ImportArray Module
>>> a = arr.array('i'，[1，2，3]) # Createarray a with integers
>>> b = arr.array('i'，[1，1，1]) # Createarray b with integers
>>> c = sum(a + b) # Create sum ofall array values
>>> print(a，b，c) # and output it
array('i'， [1， 2， 3]) array('i'，[1， 1， 1]) 9 # output>>>
You can use numpy (under Linux) or umatlib library, or even more libraries.
Figure 5 The user interface of MaixPy IDE
06
Install MaixPy IDE
Use MaixPy IDE development environment, development and debugging software will be very convenient. You can develop and debug on Maixduino, and load and execute Python programs. In addition, as shown in Figure 5, tools for image analysis are also provided. The installation process is as follows: download maixpyide linux x86_64 0.2.4installer-archive.7z or higher from the website link [4], copy it to the folder of your choice; unzip it with the command tar maixpy idelinux x86_64 0.2.4 installer archive.7z, Go to the new folder maixpy ide linux x86_64 0.2.4 installerarchive, enter and execute the following commands:
./setup.sh
./bin/maixpyide.sh
The IDE starts immediately, and all the tools used to implement the AI model are started. Next, you can test the facial recognition program.
07
face recognition
The trained AI model is used for face recognition. The AI model has analyzed the features of thousands of faces, and the neural network (NN) weights used have been adjusted accordingly. The model can be downloaded from the website link [5], the name is face_model_at_0x30 0000.kfpkg. The AI framework Yolo2 is used here for development, which divides the image object into several regions and analyzes them separately to obtain a higher recognition rate. The AI model is packaged in kfpkg format and must be downloaded to Maixduino's address 0x30 0000. Can be programmed with Kflash. Find the corresponding file and use the parameters in Figure 4 to program it to the board. MaixPy IDE is used to edit Python scripts. Under this IDE, you can develop and debug programs and download them to Maixduino.
Figure 5 includes three windows: an editor, which displays program input through syntax highlighting; a terminal, which displays program output; image analysis, which performs image display and divides the spectrum into red, green, and blue displays. There are two important buttons at the bottom left: "paperclip" is used to establish (green) or disconnect (red) the connection with Maixduino through the port "ttyUSB0". The green triangle below is used to start the script. After starting, the button will turn into a red dot with an "x" to stop the program.
For the test, I printed the faces of two famous people (Albert Einstein and Rudy Waller) and fixed them on the wall. This choice is purely coincidental, and it is said that the two faces are somewhat similar. When these photos are taken, these faces will be recognized immediately and marked with a frame. Make sure that the image is shot in the landscape as shown in Figure 6, otherwise the recognition rate will drop significantly.
Figure 6 Test setup for facial recognition
The program Face detect.py can be downloaded on the Elektor website [6]. Its simplicity once again proves the performance of the library. First, integrate and initialize the libraries required by the camera, LCD, and KPU. Then, load the neural network (NN) into the KPU starting from address 0x30 0000. The AI model is initialized by the command kpu.init_yolo2, and other constants are transmitted to set the accuracy and optimization options. The image classification is carried out in the while loop, and the image is acquired and fed to the neural network (NN). If a face is detected, the variable i records the coordinates and size of each face marker frame, and then draws it on the image. Finally, output the image (on the LCD panel) and mark data (on the serial port). For more detailed information about the KPU command, please refer to the website link [7].
In MaixPy IDE, the picture is displayed in the upper right corner, and the corresponding chromatogram is displayed below. If this information is not needed, you can deactivate the button to close the image window on the right. For better processing, mount Maixduino and LCD on the small board, and aim the camera at the front (see Figure 6). This makes it easy to capture and analyze real faces, print images or screen content. The display content can be seen on the LCD panel of Fig. 7, and the similarities of the personnel are not shown.
Figure 7 Display content on Maixduino LCD
How is the implementation behind the Yolo2 model? The neural network has 24 convolutional layers and 2 fully connected output layers (see Figure 8). There are some maxpool layers between them as filters to eliminate complexity and reduce "memory". It is worth noting that the 3×3 window is mainly used for detail recognition. To be precise, this is supported by the KPU hardware, which allows Maixduino to support such tasks to the utmost extent. Other known neural network (NN) structures even have hundreds of layers with feedback paths or other additional functions. There is no upper limit for creativity in this field, which depends to a large extent on the computing power of the computer.
08
Keep going!
The powerful hardware environment and software environment show that Maixduino is very suitable for artificial intelligence. Due to low power consumption, it is also very suitable for use in already trained neural network mobile devices. I will show you how to develop, train and execute your own neural network in the follow-up. The interface with developers is the AI framework Keras, also known as the "Lego build kit". You will also learn how to program on the ESP32 development board, such as analog acquisition. Stay curious and keep going!
Figure 8 Network architecture of facial recognition (Source: https://bit.ly/3cK3DUR)
Related Links
[1]AI for Beginners，Elektor May&June 2020: www.elektormagazine.com/200023 02
[2]Kflash:https://github.com/sipeed/kflash_gui/releases
[3]Maixduino Firmware:http://dl.sipeed.com/MAIX/MaixPy/release/master/
[4]MaixPy IDE:http://dl.sipeed.com/MAIX/MaixPy/ide/_/v0.2.4/maixpy-ide-linux-x86_640.2.4-installer-archive.7z
[5]AI models:http://dl.sipeed.com/MAIX/MaixPy/model
[6] Project software: www.elektormagazine.com/200023B 04
[7]KPU Commands:https://maixpy.sipeed.com/en/libs/Maix/kpu.html
This article is authorized to come from Elektor Media Group, our partner. If you want to subscribe to Elektor's English online content for free, please visit https://www.elektormagazine.com/.
1. It is said that many software engineers envy hardware engineers
2. Several communication interfaces commonly used in microcontrollers, such as I2C, SPI, UART, etc.
4.5 yuan changed to 70, hey, the chip is out of stock again
5. How does the RISC-V processor design the instruction set? What's so special
6. Macro definitions commonly used by embedded engineers