We currently live in a world surrounded by machine learning models. Throughout the day, you use these models more than you realize. Everyday tasks like browsing social media, taking pictures, checking the weather, etc. all rely on machine learning models. You might even see this blog because a machine learning model recommended it to you.
We all know that training these models is computationally expensive. But running inference on these models is also computationally expensive most of the time. The speed at which we use machine learning services, we need computing systems fast enough to handle it. Therefore, most models run on large data centers with clusters of CPUs and GPUs (and in some cases even TPUs).
bigger is not always better
When you take a photo, you want the magic of machine learning to happen instantly. You don't want to wait for the image to be sent to a data center where the image is being processed and sent back again. In this case, you want the machine learning model to run locally.
When you say "Alexa" or "OK, Google," you want the device to respond to you right away. Wait for the device to send your voice to the server which processes it and retrieves the information. This takes time and detracts from the user experience. Again, in this case, you want the machine learning model to run locally.
What is TinyML
TinyML is an area of study in machine learning and embedded systems that explores the types of models that can run on small, low-power devices such as microcontrollers. It enables low-latency, low-power, and low-bandwidth model inference on edge devices. A standard consumer CPU consumes between 65 watts and 85 watts, a standard consumer GPU consumes between 200 watts and 500 watts, and a typical microcontroller consumes around milliwatts or microwatts. This roughly reduces power consumption by a factor of a thousand. This low power consumption allows TinyML devices to run unplugged on batteries for weeks, months, and in some cases years, while running ML applications at the edge.
The future of machine learning is tiny but bright
Advantages of TinyML
- Low latency: Since the model runs at the edge, there is no need to send data to the server to run inference. This reduces output latency.
- Low Power Consumption: As we discussed earlier, microcontrollers have very low power consumption. This allows them to run for a long time without recharging.
- Low Bandwidth: Less internet bandwidth is used since data doesn't have to be constantly sent to the server.
- Privacy: Since the models run at the edge, your data is not stored in any servers.
Applications of TinyML
By summarizing and analyzing data at the edge of low-power devices, TinyML offers many unique solutions. Although TinyML is an emerging field, it has been used in production for many years. "OK Google", "Alexa", "Hey Siri" wake words are an example of TinyML. Here, the device is always on and is analyzing your voice to detect the wake word. I'll add some more uses of TinyML here.
- Predictive Maintenance in Industry: Machines are prone to breakdowns. Using TinyML on low-power devices, machines can be continuously monitored and failures predicted in advance. This type of predictive maintenance can result in significant cost savings. Australian startup Ping Services has launched an IoT device that automatically monitors wind turbines by attaching magnetically to the outside of the turbines and analyzing detailed data at the edge. The device can even alert authorities to potential problems before they occur.
- Healthcare: The Solar Scare Mosquito project uses TinyML to curb the spread of mosquito-borne diseases like dengue, malaria, Zika, chikungunya and more. It works by detecting mosquito breeding conditions and agitating the water to prevent mosquitoes from breeding. It runs on solar power so it can run indefinitely.
- Agriculture: The Nuru app helps farmers detect diseases in plants by taking photos using TensorFlow Lite to run machine learning models on-device. Since it runs on the device, no internet connection is required. This is a crucial requirement for remote farmers who may not have proper internet connectivity.
- Marine life conservation: Machine learning-based smart devices are used to monitor whales in real-time in waterways around Seattle and Vancouver to avoid whale attacks in busy shipping lanes.
How do I get started?
- Hardware: Arduino Nano 33 BLE Sense is the suggested hardware for deploying machine learning models at the edge. It contains a 32-bit ARM Cortex-M4F microcontroller running at 64MHz with 1MB of program memory and 256KB of RAM. This microcontroller provides enough horsepower to run TinyML models. The Arduino Nano 33 BLE Sense also includes color, brightness, proximity, gesture, motion, vibration, orientation, temperature, humidity and pressure sensors. It also contains a digital microphone and a Bluetooth Low Energy (BLE) module. This sensor kit is sufficient for most applications.
- Machine Learning Frameworks: Only a few frameworks can meet the needs of TinyML. Among them, TensorFlow Lite is the most popular and has the most community support. Using TensorFlow Lite Micro, we can deploy models on microcontrollers.
- Learning resources: Since TinyML is an emerging field, there are not many learning materials currently available. But there is also some excellent material, such as Pete Warden and Daniel Situnayake's book "TinyML: Machine Learning with TensorFlow Lite on Arduino and Ultra-Low Power", Harvard's Vijay Janapa Reddi's TinyML course, and Digikey's blog and videos on TinyML .
in conclusion
Microcontrollers are everywhere, and they collect huge amounts of data. With TinyML, we can use this data to build better products