Article Directory
foreword
Personally, I think the biggest difference between K210 and openmv is that you can run models. Although you can’t run yolov5 to get such large models, you can still use some lightweight models such as yolov2 and Mobilenet V1.
1. Introduction of MaixHub functions
Want to train a usable model, but the computer graphics card is not strong enough, it doesn't matter, we can use Spieed's MaixHub model online training to train the model. The Maixhub model training platform helps you quickly train the AI model you want. It does not require any training environment to build and code to run. You only need to select the training type, upload (mark) the data set, and Maixhub will automatically perform the training. Download it after the training is over. .
MaixHub currently has two types of training:
1. Target classification
Recognize the type of the picture, only the type of the picture is recognized, and the coordinates of the recognized object are not displayed (not framed), for example: 2.
Target detection
can recognize both objects and The coordinates of the recognized objects can be output (boxed), for example:
2. How to use Maixhub model training
First of all, you need to determine which model you need to train. If you just want to recognize objects, then first classify the objects. If you want to recognize the object category and output the recognized object coordinates, then choose object detection.
Determine the category . Including the number of categories, specific categories. For example, here is an example of identifying red balls and toys:
so there are two categories: ball and toy, which we also call labels (label),
pay attention! ! ! The category name (label/label) can only use English characters and underscores
Determine the resolution : the following resolutions are currently supported by Maixhub, other resolutions will fail training (recognition accuracy is higher when using the recommended resolution):
Target class: 224x224 (recommended)
Object detection: 224x224 (recommended), 240x240
Determine the number of data sets collected : In addition, the requirements of Maixhub must also be met:
Target category: The number of pictures in each category is not less than 40, for example, 200 pictures are collected
Target detection: The number of pictures of each type is not less than 100, such as collecting 200 pictures,
the upper limit is 2000 pictures
Finally, note that the uploaded zip file cannot exceed 20MiB
3. Picture collection
There are several ways to collect pictures:
1. Use the development board to collect the SD card, and directly collect it into the required resolution ( recommended )
2. Use the ready-made picture and use the preprocessing tool to process it into the required resolution. Note that after processing Finally, you must manually check whether the data meets the requirements, otherwise it may affect the training
accuracy training accuracy
Use the development board for data collection
Because we have understood the camera principle of MAIX BIT before, so here we mainly use the development board to collect pictures.
Use this script to capture pictures
#根据你的开发板修改摄像头和屏幕配置， 比如lcd.rotation
#准备一张支持 SPI 模式的 SD 卡， 分区为 MBR （msdos）， 格式化为 FAT32
#将目录下的boot.py文件拷贝到 SD 卡根目录
#开发板断电， 将SD卡插入开发板
#开发板上电， 程序会自动创建一个目录cap_images_1, 下次上电会创建cap_images_2， 这样就避免了覆盖
#采集一个分类的图片 按开发板上的boot按键，然后松开按键来采集一张图片， 这会将图片保存到cap_images_1/0/0.jpg， 采集的图片的名字会自动增长， 比如0.jpg 1.jpg ...
#长按boot按键切换类别目录 这会创建一个新目录，比如cap_images_1/1/, 后面采集的图片都会被保存到这个新的目录， 比如cap_images_1/1/0.jpg
#开发板断电，取出SD卡插到电脑， 打开文件管理器就可以看到采集的图片了
import sensor, lcd
from Maix import GPIO
from fpioa_manager import fm
from board import board_info
import os, sys
import time
import image
#### image size ####
set_windowing = (224, 224)
#### sensor config ####
sensor.reset(freq=22000000, dual_buff=False)
sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.QVGA) # 320x240
try:
sensor.set_jb_quality(95) # for IDE display quality
except Exception:
pass # no IDE support
if set_windowing:
sensor.set_windowing(set_windowing)
sensor.skip_frames()
#### lcd config ####
lcd.init(type=1, freq=15000000)
lcd.rotation(2)
#### boot key ####
boot_pin = 16 # board_info.BOOT_KEY
fm.register(boot_pin, fm.fpioa.GPIOHS0)
key = GPIO(GPIO.GPIOHS0, GPIO.PULL_UP)
######################################################
#### main ####
def capture_main(key):
def draw_string(img, x, y, text, color, scale, bg=None , full_w = False):
if bg:
if full_w:
full_w = img.width()
else:
full_w = len(text)*8*scale+4
img.draw_rectangle(x-2,y-2, full_w, 16*scale, fill=True, color=bg)
img = img.draw_string(x, y, text, color=color,scale=scale)
return img
def del_all_images():
os.chdir("/sd")
images_dir = "cap_images"
if images_dir in os.listdir():
os.chdir(images_dir)
types = os.listdir()
for t in types:
os.chdir(t)
files = os.listdir()
for f in files:
os.remove(f)
os.chdir("..")
os.rmdir(t)
os.chdir("..")
os.rmdir(images_dir)
# del_all_images()
os.chdir("/sd")
dirs = os.listdir()
images_dir = "cap_images"
last_dir = 0
for d in dirs:
if d.startswith(images_dir):
if len(d) > 11:
n = int(d[11:])
if n > last_dir:
last_dir = n
images_dir = "{}_{}".format(images_dir, last_dir+1)
print("save to ", images_dir)
if images_dir in os.listdir():
img = image.Image()
img = draw_string(img, 2, 200, "please del cap_images dir", color=lcd.WHITE,scale=1, bg=lcd.RED)
lcd.display(img)
sys.exit(1)
os.mkdir(images_dir)
last_cap_time = 0
last_btn_status = 1
save_dir = 0
save_count = 0
os.mkdir("{}/{}".format(images_dir, save_dir))
while(True):
img0 = sensor.snapshot()
if set_windowing:
img = image.Image()
img = img.draw_image(img0, (img.width() - set_windowing[0])//2, img.height() - set_windowing[1])
else:
img = img0.copy()
# img = img.resize(320, 240)
if key.value() == 0:
time.sleep_ms(30)
if key.value() == 0 and (last_btn_status == 1) and (time.ticks_ms() - last_cap_time > 500):
last_btn_status = 0
last_cap_time = time.ticks_ms()
else:
if time.ticks_ms() - last_cap_time > 5000:
img = draw_string(img, 2, 200, "release to change type", color=lcd.WHITE,scale=1, bg=lcd.RED)
else:
img = draw_string(img, 2, 200, "release to capture", color=lcd.WHITE,scale=1, bg=lcd.RED)
if time.ticks_ms() - last_cap_time > 2000:
img = draw_string(img, 2, 160, "keep push to change type", color=lcd.WHITE,scale=1, bg=lcd.RED)
else:
time.sleep_ms(30)
if key.value() == 1 and (last_btn_status == 0):
if time.ticks_ms() - last_cap_time > 5000:
img = draw_string(img, 2, 200, "change object type", color=lcd.WHITE,scale=1, bg=lcd.RED)
lcd.display(img)
time.sleep_ms(1000)
save_dir += 1
save_count = 0
dir_name = "{}/{}".format(images_dir, save_dir)
os.mkdir(dir_name)
else:
draw_string(img, 2, 200, "capture image {}".format(save_count), color=lcd.WHITE,scale=1, bg=lcd.RED)
lcd.display(img)
f_name = "{}/{}/{}.jpg".format(images_dir, save_dir, save_count)
img0.save(f_name, quality=95)
save_count += 1
last_btn_status = 1
img = draw_string(img, 2, 0, "will save to {}/{}/{}.jpg".format(images_dir, save_dir, save_count), color=lcd.WHITE,scale=1, bg=lcd.RED, full_w=True)
lcd.display(img)
del img
del img0
def main():
try:
capture_main(key)
except Exception as e:
print("error:", e)
import uio
s = uio.StringIO()
sys.print_exception(e, s)
s = s.getvalue()
img = image.Image()
img.draw_string(0, 0, s)
lcd.display(img)
main()
After collecting the pictures, copy the pictures in the SD card to the computer, organize them into the following directory structure, and the resolution of all pictures is 224x224
4. Label the data set (skip this step for target classification)
Note : Make sure the resolution is correct before labeling
There are two labeling methods, one is vott (click here to download), the other is labelimg (click here to download). Here I mark mainly with labelimg (note: the path of labelimg cannot have Chinese, otherwise it will crash).
After saving, a file in xml format (that is, PascalVOC format) will be generated, and each image corresponds to an xml file. After completion, the directory structure and files are as follows:
or two-level directory
5. Pack the dataset
Package the previously processed data set, use zip compression, other formats are not supported for now, and the file should not exceed 20MiB.
Target category:
one category per folder, the category name (label/label) is the folder name
Target detection: the
output zip file structure of labelimg, create a new labels.txt, input the label of the label, one per line (this is necessary, otherwise the data is invalid), such as here: then the directory structure is as follows or two-level
labelimg
output
6. Upload the dataset for training
If the progress is always 0%, you can refresh it. After the training is completed, click Download to download the training file. The downloaded file is a zip compressed file. Read the README.txt carefully after decompression. It uses Chinese and English to explain the usage method. The default is to copy all the resulting files to the root directory of the SD card if there is the latest version of the firmware. Plug it into the development board, and then power it on to run. (Note: The format of the submitted data set must be striven for, otherwise it will report an error and cannot be trained)
Summarize
The above is the general process of MaixHub's online training model. Overall, it is relatively simple. It is recommended to conduct object classification training first and then object detection training.
Frequently asked questions:
1. After the object classification training is completed, run the training file and find that the object is recognized, but the names of other objects are displayed. At this time, you can change the order of the labels in boot.py.
labels = ["Recyclable_waste_out", "Other_waste", "Kitchen_waste", "Hazardous_waste_out"]
2. ...to be added...
