IoTOctober 22, 202612 min readBy IoT Insights Hub Editorial Team

What Is TinyML? Ultimate Guide with 7 Real-World IoT Uses

Most people picture AI as a data centre full of servers. Now ask what is TinyML and the picture shrinks to a chip smaller than a fingernail, running on a coin cell, listening for one word. It is a quiet corner of AI, but it sits inside a growing number of sensors, wearables and appliances.

This guide from IoT Insights Hub explains how TinyML works, how it differs from edge AI, what hardware and tools it uses, seven real uses and where it falls short. Where a figure is old or comes from a vendor, the text says so.

New to connected devices? Start with our beginner guide to IoT. TinyML is one way to add intelligence to those devices, and our AIoT explainer shows the bigger picture.

What is TinyML and why does it matter?

Per Wikipedia, running models on embedded systems gained traction in the late 2010s as compression, quantization and efficient network designs improved. It says the term was popularised around 2019 with the book TinyML by Pete Warden and Daniel Situnayake and the creation of the TinyML Foundation. Its own citation for the book lists 2020, so treat the exact year loosely.

Arm’s TinyML blog, dated 6 May 2021, gives the most vivid definition: workloads on microcontrollers “no bigger than a grain of rice” that draw only a few milliwatts. A MITRE report by Stanislava Soro from September 2020 puts it as deep learning on microcontroller-based devices working at milliwatt power levels or below.

Why does that matter? Arm lists four benefits. Local processing avoids cloud round trips, so latency drops. Data that stays on the device is less exposed. Devices lean less on congested networks. And less cloud traffic cuts energy use. In IoT Insights Hub’s view, the last point is the quiet winner for battery-powered sensors in places nobody wants to visit twice.

The scale of the hardware base explains the interest. Arm quotes IDC saying 28.1 billion microcontrollers were sold in 2018, with 38.2 billion forecast for 2023. That forecast is old, so read it as a sense of scale and not a current count. Soro’s 2020 report says about 250 billion microcontrollers were reportedly deployed, a figure it does not source in the excerpt we read.

How does TinyML work?

The usual TinyML workflow: train elsewhere, compress, then run locally.

Soro’s report says training still happens in the cloud or on more powerful hardware, and calls on-device training a future need. A 2024 review by Ji Lin, Song Han and colleagues, “Tiny Machine Learning: Progress and Futures,” covers research that goes further, including on-device training and a system called MCUNet for ImageNet-scale tasks on IoT devices. The review’s closing line is that “today’s large model might be tomorrow’s tiny model.”

The shrinking step does most of the work. Soro lists pruning, parameter quantization, Huffman coding (reported to save 20 to 30% of parameter storage) and knowledge distillation. Arm explains that models often swap floating-point maths for simpler 8-bit operations to cut processing and memory needs. XenonStack, a vendor, describes quantization as turning 4-byte floats into 1-byte integers.

Model choice is a trade-off between size and accuracy. Soro says simple jobs such as voice activity detection can use small fully connected networks, while speech recognition needs much larger recurrent models. That is why most TinyML demos pick narrow jobs: one wake word, one vibration pattern, one yes or no.

The decision then has to go somewhere. A TinyML node rarely works alone, and its output usually travels over a low-power link. For the options, see our guide to energy-efficient IoT protocols and our explainer on what is MQTT, which many gateways use to forward results.

What is the difference between TinyML, edge AI and cloud AI?

The table below uses Arm, MITRE, Google and a July 2026 comparison from Blockchain Council, which sells AI certifications and gives no sources for its figures. The cloud column and the final row are IoT Insights Hub’s own summary.

MITRE gives the clearest hardware numbers: typical microcontrollers have 192 to 512 KB of SRAM and 256 KB to 2 MB of flash storage. Those are 2020 figures, and newer chips may offer more, but the order of magnitude is the point. A gateway or phone has far more working memory.

So is TinyML a subset of edge AI? In IoT Insights Hub’s view, yes: it is edge AI at the smallest scale. If the job needs a camera feed and a large model, you want a bigger edge device. Our guide to edge AI benefits covers that wider tier, and IoT edge computing explained shows where it sits in a system.

When should a team choose the edge over the cloud at all? IoT Mail Bridge sets out seven signs in its guide on when to use edge computing, and its piece on the top 5 ways edge AI transforms IoT is a useful companion.

What hardware and tools does TinyML use?

Google’s documentation says the LiteRT runtime is built for devices with only a few kilobytes of memory, with a core that fits in 16 KB on an Arm Cortex-M3. It needs no operating system, no standard C or C++ libraries and no dynamic memory allocation. It is written in C++17, needs a 32-bit platform and is open source.

Google also lists example projects: a micro speech demo that detects the words “yes” and “no” from a microphone, and a person detection demo that uses camera data. These make a good first test, because they show a real model running in a few tens of kilobytes.

The Pico price shows why hobbyists and startups can try this at low cost. Arm gave about $4 in 2021, so check current pricing before you buy. Building a TinyML product still takes real work, and for the business side of building in AI, The Business Perspective tracks the money in its Q3 2026 venture funding analysis, which puts AI at 64% of funding in its headline finding.

What is TinyML used for? 7 real uses

1. Wake-word and keyword spotting. Arm says an early speech-wake model ran on a Cortex-M3 with 15 KB of code and 22 KB of data. Google’s micro speech example detects the words “yes” and “no” from microphone input on a small board.
2. Machine health monitoring. Arm lists industrial machines that predict their own maintenance needs, and Soro names industrial monitoring as a TinyML application. XenonStack, a vendor, describes predicting equipment failures and alerting workers when service is due.
3. Person detection. Google’s LiteRT person detection example uses camera data to decide whether a person is present. Soro lists surveillance, with person or object detection, among the applications of TinyML.
4. Wearables and vital-sign monitors. Arm says healthcare monitors can track vitals while maintaining privacy, because data stays on the device. Soro also lists healthcare wearables as a TinyML application.
5. Crop and livestock sensing. Arm gives crop sensors that detect damaging insects as an example. XenonStack, a vendor, adds cattle wearables that track vital signs, so treat that one as a vendor claim.
6. Smart retail shelves. Arm describes in-store shelves that request restocking. XenonStack also mentions monitoring shelves and warning when stock runs low.
7. Traffic and city sensing. Arm gives responsive traffic lights that reduce congestion as an example. XenonStack cites real-time traffic sensing for routing, attributed to Swim.AI. Both are examples, not audited deployments.

Across all seven, what is TinyML doing? In IoT Insights Hub’s reading, it turns a stream of raw sensor data into one small decision, close to where the data appears. For a wider look at how that fits with connected systems, see our piece on AI and IoT creating new value, and for the maintenance angle read predictive maintenance and how it reduces downtime.

A local decision is only useful if someone acts on it. When a TinyML sensor flags a fault, the alert still has to reach a person. IoT Mail Bridge explains how to avoid drowning teams in notices in its guide to IoT alert fatigue and seven fixes for factory teams.

What are the limits of TinyML?

• Tight memory: Soro names small memory and short battery life as the two key obstacles to running neural networks on microcontrollers, and the 2024 review calls the memory “tiny.”
• Accuracy loss: XenonStack says compression can reduce accuracy, so teams must balance size against performance.
• Limited operations: Google says LiteRT supports only a subset of TensorFlow operations and a limited set of devices.
• No on-device training: Google says LiteRT for Microcontrollers does not support it, though research such as the 2024 review explores it.
• Low-level code: Google describes a low-level C++ API with manual memory management, which raises the skill bar.
• Fragmented hardware: XenonStack says models often must be recompiled for each device, and the 2024 review notes limited compiler and inference-engine support for bare-metal devices.
• Updates and security: Soro lists difficult model updates on remote devices, and XenonStack raises privacy and security concerns for over-the-air updates.
• Weak benchmarks: Soro (2020) points to a shortage of standard datasets and baseline models, and mentions TinyMLPerf. That gap may have narrowed since, but we could not confirm.
• Source bias: XenonStack and Blockchain Council both sell services or training, and some of their claims, such as a year of battery life on a coin cell, are unsourced.

None of this means TinyML is a gimmick. It means the job has to fit the chip. If you want a camera, a large language model or frequent retraining, a bigger edge device or the cloud is the honest answer. Soro also suggests non-neural methods such as decision trees and support vector machines for some cases, since they can need fewer resources.

Connectivity choices matter as much as the model. Compare short-range options in our guide to Zigbee vs LoRaWAN, and read our guide to IoT security risks and how to fix them before you ship updates over the air.

TinyML FAQs

Is TinyML worth learning?

If you build or buy connected devices, yes. It is the practical way to put a small, always-on decision into a battery-powered sensor without sending every reading to the cloud. Start with a narrow job, expect to compress and test a lot, and keep a bigger device in reserve. If you remember what is TinyML in one line: a small trained model making local decisions on a tiny chip.

IoT Insights Hub will keep publishing practical, sourced guides on IoT, edge AI and connected hardware. For the funding side of building in this space, read The Business Perspective’s guide to raising seed funding.