Maix Bit (K210) Nanny Level Getting Started Tutorial Series
Maix Bit (K210) Nanny Level Getting Started Tutorial—Environment Construction
Maix Bit (K210) Nanny Level Getting Started Tutorial—Basic Use of Peripherals
This is a series of K210 quick start articles. The main content is to introduce the use of MaixHub, an online training model, and how to deploy it to K210.
The premise of reading this article: the reader has a certain understanding of basic supervised learning, and has a certain concept of learning rate, number of iterations, network model, etc. If not, add it yourself or click here to read the official documents to learn the basics ~
Most of the content of this article comes from official documents, the purpose is to help readers quickly get started training their own models, infringement deleted!
Article Directory
1. K210 hardware introduction
In general, there are two ways to run a neural network model. One is to run directly through the CPU, and the other is to accelerate the operation through the KPU or GPU. K210 uses KPU to accelerate the operation of the network model, making its operation speed faster.
1. Memory introduction
The storage medium in MaixPy is mainly composed of Flash and SD card, which are divided into three areas, namely MaixPy.bin firmware area, xxx.kmodel model area, file system area: spiffs (SPI Flash File System) on Flash, SD card It is Fatfs (FAT file system).
It usually starts at 0x300000. The reason why the model files are not burned in the Flash file system is as follows:
The file system in Flash does not have enough memory for large models, larger models can be placed on SD cards. Reading model files directly is faster than reading through the file system.
But there are also the following reasons to burn the model into the flash, that is, the model itself is relatively large , so it needs to be burned into the flash instead of the SD card. Because the model is large, the SD card may have insufficient memory when loading the model.
Memory management:
In MaixPy, two kinds of memory management are currently used, one is GC (garbage collection), and the other is system heap memory, both exist at the same time.
For example: the chip has 6MiB of memory, adding the firmware uses the previous 2MiB, leaving 4MiB, the default GC uses 512KiB, and the rest is for the system heap memory management.
Note:
The total size of the GC memory can be set, so the size of the GC memory can be appropriately modified according to the specific usage, for example:
In order to load a larger model, you can set the GC memory to be smaller.
If allocating new variables indicates insufficient memory, you can set the GC memory to be larger. If it is not enough, it is necessary to consider reducing the size of the firmware, or optimizing the code.
2. KPU introduction
KPU is a general-purpose neural network processor, which can realize convolutional neural network calculations with low power consumption, obtain the size, coordinates and types of detected targets from time to time, and detect and classify faces or objects. The KPU realizes the hardware acceleration of the four basic operations of convolution, batch normalization, activation, and pooling, but they cannot be used separately and are integrated acceleration modules. There are also some restrictions on KPU usage.
memory limit
- The K210 has 6MB general RAM and 2MB KPU dedicated RAM. The input and output feature maps of the model are stored in 2MB KPU RAM. Weights and other parameters are stored in 6MB general RAM.
Acceleration Limit
There are many acceleration restrictions. Some operators can be fully accelerated, while others cannot. You just need to know this. Specifically which ones are acceptable and which ones are not allowed to go to the official website to check~
2. Introduction of MaixHub
The MaixHub website provides model training and model sharing functions. You only need to prepare the data sets that need to be trained. You don’t need to build a training environment and code. You can quickly train the model by uploading the training data, which is convenient for quickly making your AI applications, or getting started. The process and principle of AI training.
3. Get image data
1. Get online
To obtain image data, you can obtain it yourself with a crawler, or you can obtain it from the actual running development environment. If it is obtained from the actual running environment (it is recommended to use this method to train with the pictures obtained by yourself, the accuracy will be higher), you can refer to the following code. If it is obtained by crawlers, you can refer to
Baidu crawling pictures
2. Obtain offline
Here I wrote a simple code to get data from the camera, you can refer to it. Currently supports two ways to acquire pictures, timing acquisition or button acquisition.
However, due to my limited personal level, it is impossible to solve the problem that the directory created by using uos.mkdri() in python cannot be deleted normally using python (it is recommended to modify the content of the dictionary through the relevant interface, because the created directory cannot be deleted normally, unless it is on a PC operate).
If so 目录或者文件异常, it may be caused by not closing the file properly. You can delete the created things on the PC.
Note that SDa card is required, and the files are all operated on the SD card.
For the basic process, first call init() to create an index file. After that, the corresponding directory will be created to store the files according to the content of the dictionary. For example: the format of the dictionary created by init() is as follows.
The directories /pic/sit/right and /pic/sit/error will be automatically created, and then the value of right or error will be automatically read and saved as the picture noun.
dictx={
'sit': {
'right': 0, 'error': 0}, 'object': {
'portable_battery': 0, 'Instant_noodles': 0,'mouse': 0}}
If the Maix Bit board is used, the following code can be run directly: the key is the key on the development board
from machine import Timer
from Maix import GPIO
from fpioa_manager import fm
import sensor, lcd
import uos,sys
def create_dir(dir,chdir='/sd'):
try:
if(uos.stat(dir)):
print("PATH existed")
except:
uos.mkdir(dir)
# 函数整个字典读取,文件默认名字是index.txt
def read_pic_index(file_path='/sd/pic'):
uos.chdir('/sd/pic')
file = open('index.txt','r+',encoding='utf-8')
index= eval(file.read()) #读取的str转换为字典
file.close()
return index
# 添加一个字典,第一次创建，写key=='none'
def add_pic_index(key,dict_value,file_path='/sd/pic'):
if(key!='none' ):
index = read_pic_index(file_path)
file = open('index.txt',"w+",encoding='utf-8')
index[key]=dict_value
file.write(str(index)) #把字典转化为str
file.close()
else:# 第一次创建
create_dir('pic',file_path)
uos.chdir(file_path)
#print(uos.getcwd())
file = open('index.txt',"w+",encoding='utf-8')
file.write(str(dict_value)) #把字典转化为str
file.close()
# 删除一个字典
def del_pic_index(key,file_path='/sd/pic'):
import sys
global index # 声明全局索引
index = read_pic_index('index.txt')
file = open('index.txt','w+')
try:
del index[str(key)]
except KeyError:
print('key :'+key +' have been del or without this key')
#file.close()
else:
pass
file.write(str(index)) #把字典转化为str
file.close()
# 打印所有的索引
def print_pic_index(file_path='sd/pic'):
cat_index=read_pic_index('index.txt')
for key in cat_index:
print("key:%s value:%s"%(key,str(cat_index[key])))
# 获取key的索引
def get_pic_key_index(key,file_path='sd/pic'):
cat_index=read_pic_index('index.txt')
for fkey in cat_index:
if fkey==key:
return cat_index[key];
# 设置某一个key的index
def set_pic_key_index(key,dict_value,file_path='sd/pic'):
add_pic_index(key,dict_value,file_path)
# 清楚所有的index
def clear_pic_index(file_path='sd/pic'):
cat_index=read_pic_index(file_path)
for key in cat_index:
del_pic_index(key,file_path)
# 这里解释以下，init()创建一个索引文件，保存标签图片
# 比如，/sd/pic/index.txt中的数据就是dictx的数据，使用前需要先调用init()
# 存放图片也跟这个有关系，存放图片的路径会是，/sd/pic/sit/right或者
# /sd/pic/sit/error
def init():
# init()执行一次就行了，之后屏蔽掉就OK
dictx={
'sit': {
'right': 0, 'error': 0}, 'object': {
'portable_battery': 0, 'Instant_noodles': 0,
'mouse': 0}}
try:
if uos.stat('/sd/pic'):
print("PATH existed")
except:
uos.mkdir('/sd/pic')
add_pic_index('none',dictx)
index=read_pic_index()
print(index)
# 初始化摄像头
def sensor_init():
sensor.reset()
sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.QVGA)
sensor.run(1)
sensor.skip_frames()
# 获取一帧图片
def get_frame_pic():
img = sensor.snapshot()
return img
# 获取获取图片计数和图片设置路径
pic_num=1
save_path=''
# 保存图片设置
def save_pic(timer):
global pic_num
global save_path
img=get_frame_pic()
lcd.display(img)
file_name=save_path+'/'+str(pic_num)+'.jpg'
print(file_name)
img.save(file_name, quality=95)
pic_num+=1
#开发板上RST的按键IO
KEY=16
# 保存图片,保存的数量，1张图片拍摄的时间间隔（ms），这里使用计时器的方式
# 参数：ikey是字典类标签，iikey是某一个标签
# 参数：num是保存图片数量，file_index_path就是标签保存途径
# 参数：interval是’period‘获取一张图片间隔,推荐300~500ms
# 参数：mode是模式，按键’key‘，模式’period‘
# 注意：使用按键的方式，interval将不再起作用
def start_obtain(ikey,iikey,file_index_path='/sd/pic/index.txt',num=100,interval=300,mode='period'):
global pic_num
global save_path
print('sensor_init & lcd')
pic_index=get_pic_key_index(ikey,file_index_path)
print(pic_index)
sensor_init()
lcd.init()
if mode=='period':
if interval < 150:
interval = 150
# 配置定时器
print('init irq')
tim = Timer(Timer.TIMER0, Timer.CHANNEL0, mode=Timer.MODE_PERIODIC, period=interval, unit=Timer.UNIT_MS, callback=save_pic, arg=save_pic, start=False, priority=1, div=0)
elif mode=='key':
# 配置按键外部中断
fm.register(KEY, fm.fpioa.GPIOHS0)
key = GPIO(GPIO.GPIOHS0, GPIO.IN, GPIO.PULL_NONE)
key.irq(save_pic, GPIO.IRQ_RISING, GPIO.WAKEUP_NOT_SUPPORT,1)
else:
print('input mode error please input mode = period or key')
return 0
# 获取索引
pic_index=get_pic_key_index(ikey,file_index_path)
print(pic_index)
# 根据索引的进行处理
for pkey in pic_index:
if iikey==pkey:
save_path='/sd/pic/'+ikey+'/'+pkey # 保存文件目录
create_dir('/sd/pic/'+ikey) # 创建相应路径的文件夹
create_dir(save_path) # 创建相应路径的文件夹
last_num=pic_index[pkey] #获取上次已经保存到的数量
pic_num=last_num #设置当前数量
while(pic_num<num+last_num): # 等待获取图片信息完成
#print(save_path+' now has get pic_num %d'%(pic_num))
if mode=='period':
tim.start() #启动
# 更新索引
print("have done = %f"%((pic_num-last_num)/num))
pic_index[pkey]=pic_num
set_pic_key_index(ikey,pic_index) # 更新key index
if mode=='period':
tim.stop()
tim.deinit()
elif mode=='key':
key.disirq() # 禁用中断
fm.unregister(KEY)
else :
return 0
print(pkey + 'have right all')
init()
start_obtain(ikey='sit',iikey='error',file_index_path='/sd/pic/index.txt',num=500,interval=100,mode='period')
operation result:
PATH existed
sensor_init & lcd
{
'error': 101, 'right': 9}
init i2c:2 freq:100000
[MAIXPY]: find ov5642
[MAIXPY]: find ov sensor
init irq
{
'error': 101, 'right': 9}
PATH existed
PATH existed
have done = 0.000000
/sd/pic/sit/right/9.jpg
have done = 0.002000
/sd/pic/sit/right/10.jpg
have done = 0.004000
/sd/pic/sit/right/11.jpg
have done = 0.006000
/sd/pic/sit/right/12.jpg
4. Training model
The data for training the model is simply to train the classification algorithm. I did two simple trainings, both of which obtained data from Maxi bit, and trained the model for recognizing charging treasures and books. The data set is relatively casual, and it is enough as a demonstration.
I originally planned to use gitee, and then there will be restrictions on pictures and external links, so it is better to use github. If you can't access github, click the link below, which contains accelerators.
Maix Bit (K210) Nanny-level Getting Started Tutorial—Environment Construction
Data and related codes can be downloaded here:
github: Self-training model demo code
to add data sets
No need to add a verification set, just add all the training sets, the verification set can be left blank, and 10% will be automatically divided from the training set as the verification set during training.
View the uploaded training data
After uploading, you can view the uploaded data here:
Start queuing training
Select model selection nncase, other defaults are fine, click Create training task to start queuing training.
Just wait for the training to complete
The training result is as follows: because the data set is simple and the scene is single, the accuracy is indeed close to 1.
5. Deployment model
If you can't access github, click the link below, which contains accelerators.
Maix Bit (K210) Nanny-level Getting Started Tutorial—Environment Construction
Data and related codes can be downloaded here:
github: self-training model demo code
1. Normal model deployment
Click to deploy the model:
Select manual deployment, download the model, and get the compressed package, which contains the model file and reference demo file, and the training result report file.
We choose to put the model into the SD card, load the model from the SD card, import the model into the SD card, and remember to modify the reference 代码中的路径. If there are any strange errors, it is recommended to re-burn the firmware. The firmware is the mini version, and the firmware is already in the code.
if __name__ == "__main__":
try:
# main(labels=labels, model_addr=0x300000)
# 修改模型所在路径
main(labels=labels, model_addr="/sd/model/model-32464.kmodel")
except Exception as e:
sys.print_exception(e)
lcd_show_except(e)
finally:
gc.collect()
It can also be deployed to flash, and the mode of deploying to flash is the same as that of firmware loading, so it will not be demonstrated here. Select # main(labels=labels, model_addr=0x300000) to run this code, and select the model to deploy to 0x300000 and it will be OK.
When running the model, the model is incompatible. Pay attention to see if the model you choose to train matches the platform.
2. Large model deployment
If the model itself is relatively large, there may be a prompt of insufficient memory. In this case, there are generally the following three solutions.
- Modify the gc memory and shrink it
- Choose a smaller firmware, recommend the mini version
- kpu.load_flash load model
a. Modify gc memory
Click to view official documents
b. Load mini firmware
This is very simple, so I won't demonstrate it. The firmware is in the code I shared, of course it is the maix bit version.
c. kpu.load_flash loading model
The selected kpu.load_flash way to load a large model can theoretically load an infinitely large model, as long as each layer of the model is <4M, but in fact the flash of the K210 is limited, and the model can only be limited. kpu.load_flash Only supports loading from internal flash, does not support loading from file system
Before loading, we need to preprocess the kmodel model file on the PC (reverse the byte order of the model), and run the official model conversion code of maix bit. The code is in the gitee I shared.
The model conversion is carried out on the PC, remember to check whether your python version is 3.
For details, you can see the official instructions shared, which are in the shared files.
Use powershell to open, cd to switch the directory where the model is located.
Excuting an order:
- cd switch directory
cd C:\Users\13029\Desktop\Used\Online-training\code\load_big_model
python model_le2be.py demo_load_big_model.py
Burn the model, select the _be.kmodel after burning conversion
The specific code modification is only to modify the way to load the model, and other codes are the same as the way to load the model from the SD card above.
#task = kpu.load(model_addr)
task = kpu.load_flash (model_addr,1, 0xC000, 60000000)
5. References
KPU hardware foundation
deep learning foundation
Mirco py documentation
Sipeed official documentation