What is TinyML?

A guard dog does not think very hard. It sleeps most of the night. It wakes for a footstep at the gate and ignores the wind in the trees. That is a small brain doing one job well, on almost no fuel.

TinyML is the same idea in electronics. It means running machine learning on very small, cheap, low power computers, so a device can recognise something useful without sending anything anywhere.

The full name is tiny machine learning. The chips it runs on are microcontrollers. A microcontroller is a whole small computer on one chip, the sort found inside a microwave, a toy or a water meter.

How small is small

Numbers help here, so keep three rough ones in mind.

A phone holds gigabytes of memory. A microcontroller usually holds memory measured in kilobytes, which is thousands of times less. A TinyML model therefore has to fit in a space smaller than a single photograph.

Power is the second number. A laptop chip draws watts. A microcontroller doing this work often draws milliwatts, which is thousandths of a watt. That is why a device can run for months on a small battery.

Price is the third. These chips usually cost a few dollars or less. That is what makes it reasonable to put a thousand of them in a field.

How a model gets that small

Training still happens on a big machine. Nobody trains a model on a microcontroller. The small chip only runs the finished model, which is called inference.

Getting the model to fit takes three main tricks.

• Quantisation. The model stores its numbers with less detail, often as small whole numbers instead of long decimals. The model shrinks and runs faster, and it usually loses very little accuracy.
• Pruning. Parts of the model that barely affect the answer are cut away.
• Designing small from the start. The model is built for the chip it will live on, rather than shrunk down afterwards.

Stop on the word “fit”. A model that is one percent more accurate is worthless if it does not fit in the memory you have. On these chips, size decides everything.

What TinyML is good at

TinyML suits narrow jobs with a clear answer.

Wake words are the classic case. Your phone or speaker listens for one short phrase using a tiny model, and only then wakes the hungry part of the system.

Vibration is another. A model on a motor can learn the sound of healthy running and report when the pattern changes. That is a warning weeks before a bearing fails.

Counting and spotting also fit. Counting cars at a junction. Spotting a pest in a trap. Telling a person from a goat at a fence line, and sending one word rather than a video.

The common thread is privacy and power. The raw sound or picture never leaves the device. Only the result does.

What TinyML is bad at

Large models are out of reach. A chatbot that writes essays needs billions of stored numbers. It will never fit on a chip with kilobytes of memory.

Fine detail is hard too. A tiny vision model can say “there is a vehicle”. It cannot read the number plate reliably.

Change is the third weakness. A model trained on one factory or one crop may fail at the next site. Light, dust, accents and background noise all shift the input.

Therefore treat TinyML as a filter, not an oracle. It answers a narrow question cheaply, and hands the hard cases to something bigger.

How it fits with edge computing

TinyML is the smallest form of edge computing. Both keep the work near the data. The difference is scale.

Edge computing may mean a small server in a back room with mains power and a fan. TinyML means a battery powered chip glued to a machine or buried in a wall. Everything about the design follows from that battery.

A useful way to see it: TinyML turns a sensor into a reporter. The sensor stops sending readings and starts sending conclusions.

Where you already meet it

You meet TinyML more often than you notice. It listens for the wake phrase on a phone or speaker. It counts steps on a watch, and tells walking from cycling. It notices a fall in a hearing aid or a health band.

Outside the home, it watches machines in factories and counts people through doorways. It listens for chainsaws in forests. It checks whether a hand washing station was used. Many of these sit inside larger Internet of Things systems, and their summaries can feed a digital twin of the machine or the site.

What is coming next

Chips are getting better at this specific job. More microcontrollers now ship with a small block designed only for running models, which cuts the energy per answer.

Tools are getting simpler too. Training a small model and packing it onto a chip used to need a specialist. It is becoming ordinary engineering work.

Watch for on device learning as well. Most devices today only run a fixed model. Some are starting to adjust that model in place. A sensor can then learn the normal state of the exact machine it is bolted to.

What to try first

Do not start with the chip. Start with the sentence.

Write down the one thing you wish a device could notice. Use plain words and a yes or no answer. Someone is at the gate. The motor sounds wrong. The tank is filling. Then ask what a person would listen to or look at to decide. If a person could tell within a second, from a sound or a small picture, TinyML can probably do it too.

Just Out Tech explains new research in plain language. This article was drafted with AI assistance and checked by a human against the original source.