AI that never leaves the device. Privacy, offline capability, latency — and why that can matter categorically.
Category · AI & Agents
AI that never leaves the device.
On-device AI means the model runs on the device itself — phone, tablet, laptop, embedded device — not in the cloud. Data is processed locally, answers are generated locally. The upside is threefold: privacy, offline capability, latency.
When it makes sense.
Whenever the data the model sees is sensitive (health, finance, personal) or offline operation is a product promise. Realised via optimised runtimes like Core ML (Apple), TensorFlow Lite, ONNX Runtime or llama.cpp.
The cost: smaller models, lower performance, more engineering at the edge. That isn’t always worth it — but when it is, the difference to a cloud approach is categorical, not incremental.
