1. What is openmv?
Openmv is a machine vision module based on STM32 developed by a foreign team. Openmv is an open source project. All codes and hardware are open sourced under the MIT protocol. It can use micropython to develop upper layer application codes, aiming to be as simple as arduino programming Various image recognition functions can be realized. Although STM32 is not a dedicated image and video processor, it can still do many things with its super image processing performance as an MCU. I would like to thank the Chinese team Xingtong Technology for making great contributions to the promotion, usage and usage skills of openmv;
2. Why develop openmv independently?
First of all , the original intention of developing an openmv is that it is too fun. At the beginning, I bought several modules to play. Later, after understanding some hardware construction costs and principles, I felt that the price was really unbearable. A simple hardware circuit board and various These kinds of accessories are beyond the reach of ordinary students, which prevents many players who love learning and have limited funds; secondly
, the latest openmv source code basically does not have many useful updates in the algorithm of image recognition, except for the original color (connected domain) algorithm)/shape (Hough transform)/two-dimensional code (zbar)/face (haar_casecade)/template matching (CNN)/binary segmentation (canny operator)/linear regression/HOG feature/keypoints tracking/label recognition Commonly used image segmentation and recognition algorithms, the machine learning and deep learning frameworks added later have basically no practical application value (3-layer neural network, frame rate 4-5 frames, can only be used for demonstration); finally, why not use
traditional machines How about taking a step forward in the application of vision? In fact, all deep learning, pre-processing and post-processing are inseparable from traditional algorithms, and sometimes we need to use a combination of traditional algorithms and AI to complete a complex application. We hope to make a product that can be learned and applied in practice, rather than just a so-called profit-making method for genuine merchants (according to the MIT open source agreement, openmv has no genuine version, and buying a key is just an IDE verification and a request to the openmv team donation).
3. What has Mengfei oepnmv done?
(1) Openmv based on STM32F407
Mengfei Xiaomeng implements openmv program support on STM32F407. The official source code no longer supports the openmv1 generation, but in fact the performance of the openmv2 generation (STM32F427) except for dozens of KB of memory, other There is no difference between hardware resources and STM32F407, and F407 also supports CCM fast memory and DCMI interface, so the hardware resource allocation of F407 is rewritten according to the resource allocation of openmv2 into an omvboadrconfig.h header file and the boot address allocation in the bootload is changed, and It supports the operation of mapping the internal FLASH into a U disk, and opens the algorithm of template matching and optical flow, so that it supports most of the traditional visual algorithms at QQVGA resolution, and implements the openmv firmware on STM32F407VGT6; the code function can be equivalent to openmv3 , For this reason, I also compiled documents about openmv source code download, Linux environment installation, source code compilation, decomposition of each part, code flow and so on.
(2) Although openmv4 based
on STM32H7 implements openmv on STM32F4, its processing performance and resolution limitations still greatly reduce the playability, so I made an article on the cost-effective STM32H750, because STM32H750VBT6 has the actual use of 2MB FLASH and 1MB storage, hardware resources are consistent with H743, the only difference is that H750 consumes more power than H743, and generates a little more heat, but it does not affect its excellent processing performance and high cost performance; therefore, use the open source STM32H7_openmv Code, on which secondary development is carried out; and supporting openmv hardware is produced, which
supports OV7670 camera interface (2.54 socket interface, supports ov7670/7725/2640/5640, etc.)
8MB_QSPI_FLASH, 2-way 5V steering gear drive interface, two-way serial port, 4-way separate PWM output can control the motor, 8ping SPI LCD interface, SD card, SW download port, and can be configured into various functions through IO port multiplexing;
hardware The biggest feature is that it doesn’t need any adapter board, no SD card and external flash, which can directly save a lot of py codes
The application development based on the openmv source code specifically completes the following contents (these applications will be developed gradually in the future):
1) Support the driver of OV7670 (I checked a copy of the Linux ov7670 camera driver during the period), and change its resolution from VGA, CIF , QVGA, QCIF all resolutions support all window setting functions (QQVGA/VGA 15FPS under standard frame rate 24M clock configuration, CIF/QCIF30FPS), some ov7670 can support frequency multiplication 48MHz, and grayscale images can reach 60fps; The price of OV7725 is much lower, and the frame rate for image recognition is enough (generally, the image recognition frame rate of openmv will not exceed 30 frames); 2) It
supports the driver of ST7735 , ST7735 is a general-purpose TFT LCD screen, and the STM32 The configuration code of the LCD screen and the setting of the horizontal and vertical screens are added to the openmv source code, which can support the common 8pin interface ST7735 LCD screen, instead of buying an LCD screen with the official openmv
interface
. mnist handwritten font recognition neural network algorithm, configured and generated STM32 running code in STM32_CUBE AI, transplanted to openmv, added cubeai compiler, after successful compilation, the mnist handwritten digit recognition algorithm can be used in openmv (in fact, compared to cubeai Frameworks such as tenserflow and caffe are easier to implement neural networks, and support the underlying architecture of ARM-cortex-M4/M7. There are ready-made conversion tools on the ST official website, so it is a good choice to use CUBE_AI for some simple deep learning) 4
) Openmv routine finishing
Although the IDE comes with a lot of routines, in actual application, external sensors and devices may be needed, so I have organized many openmv routines, including ST7735/ST7789 py code driver, OV5640 auto focus, shape recognition ( circle/rectangle/triangle), color recognition tracking, face recognition tracking, ultrasonic distance measurement, image height determination distance measurement (table tennis distance measurement), license plate recognition (template matching), face comparison (LBP feature), light Flow/visual positioning, cricket system, electromagnetic gun system, infrared thermal imaging temperature measurement and many other openmv routines
5) openmv vision car
We have developed a set of smart tracked cars on openmv, using L298N drive and metal chassis mounted tracks Car, and with the SG90/MG90S gimbal, it can realize the automatic tracking of the gimbal and the simultaneous tracking of the front, rear, left, and right sides of the car, which greatly increases its playability, and the price of the whole set has reached 1/3 of the official board, which is also a huge Progress;
6) openmv robotic arm
Based on the self-developed openmv, Mengfei Intelligent has produced a three-degree-of-freedom robotic arm based on openmv, and can perform target recognition and grasping operations through the robotic arm
7) openmv WiFi image transmission module
Mengfei openmv A WiFi image transmission module is also produced, based on the development of ESP12 arduino, the image is received through SPI, and then sent out by webserver, which can be displayed on the mobile phone, VGA can achieve 10fps at the fastest, QVGA can reach 18fps, QQVGA-25fps, basically Real-time effect has been achieved, which is many times faster than AT-WINC1500. Due to the low power consumption of ESP8266, the overall module power consumption is 5V-0.25A.
8) The openmv LCD module
is based on the ST7735 LCD screen, using a common 8pin-LCD to connect to openmv, and calling the underlying functions of openmv, which can support RGB display and grayscale display
9) openmv camera adapter board
Based on the design of Mengfei openmv, it can support different ov series cameras, so a 24pin dvp to 2.54 interface camera adapter board is made, which can connect OV7670/7725/2640/5640/3640 and other dvp cameras to openmv
10) Although the openmv two-axis gimbal
has an open source 3D printing gimbal, it is too troublesome to use because of the accuracy problem and the adapter board problem, so the fpv two-axis gimbal based on the model aircraft is drilled and modified, and the SG90/MG90s steering gear is used After installation, it can be directly plugged into openmv to realize color, QR code, face tracking and other playing methods
4. What else can we do?
1) Bottom resource optimization
In fact, the processing performance of STM32H7 is far more than that, because the bottom layer of openmv uses internal SRAM as the memory space for algorithm operation. If important processing algorithms are run in TCM fast memory, the processing performance will be doubled;
2) Algorithm application
Based on the existing machine vision algorithm, we can also realize many application functions of
license plate recognition (license plate positioning, binarization segmentation, character segmentation, distortion correction, character matching, etc.), and the performance of H7 is completely sufficient for license plate recognition. Algorithms, although the processing resolution will not be very large, can be applied to roadside intelligent parking;
human detection (image binarization segmentation, corrosion, expansion, edge extraction, target rectangular frame detection, rectangular frame screening), human body extraction based on visual segmentation Target position; using MLX90640 infrared images based on temperature information will greatly improve the accuracy of human detection, which can be applied to smart home, body temperature detection, fire prevention and disaster prevention fields; lane line extraction (image filtering, canny
operator binarization , Hough transform, line segment extraction, ROI area selection, output lane line position), can be applied to automatic driving control;
industrial defect detection (using image segmentation algorithm for defect extraction and positioning algorithms). . . Wait, in fact, there are many practical applications that can be done;
Finally,,,,,, the last picture, openmv technology exchange group, 718304823, please note where to get the group number when adding the group, oh
Mengfei openmv visual car (Salted fish or Taobao search, Mengfei openmv or Mengfei smart, Go to my homepage to view)
