Edge AI / How it works
AI that runs where
your data already is.
On a server in your office, on a CPU in a closet, inside a network with no internet at all, in a phone or tablet app, on a small device in the field, or on whatever device opens our playground.
Where a model can run
From a hosted API
to no network at all.
Each step to the right keeps more under your control and asks more of your hardware. Most firms end up with a mix, chosen task by task.
GPU server on site
- Where your data goes
- Nothing leaves the building
- What fits
- Mid sized open models for a whole office
- When it makes sense
- Many people use it every day, so owning the hardware beats paying per seat.
The models we deploy
More than chat.
Every kind below runs on hardware you control. Each one has a working example in the playground.
Language models
Drafting, summaries, answering from your files, and the reasoning behind an agent. Small ones run on a CPU; larger ones want a GPU.
Try it in the playground 02Speech to text
Calls, meetings and dictation transcribed on site. Whisper sized models run faster than real time on ordinary hardware.
Try it in the playground 03Text to speech
Natural voices generated locally for phone lines, kiosks and announcements, with no audio sent to a provider.
Try it in the playground 04Realtime voice
Hear, think and speak in one loop. Local voice agents answer without a round trip to the cloud and keep working when the internet drops.
Try it in the playground 05Vision and video
Models that watch camera and video feeds next to the cameras, so footage never has to leave the site to be understood.
Try it in the playground 06Embeddings and search
Turn documents into vectors so a question finds the right passage by meaning, then answer from those passages with sources.
Try it in the playgroundEdge devices
The same models,
on every device.
Our playground opens from a link because that is the fastest way to let you try it, but the models run on your device either way. The model files are the same ones we put inside phone and tablet apps and on small devices in the field, run by ONNX Runtime on each platform.
Any device, from a link
No install. The model runs on the device itself, on its graphics chip through WebGPU or its CPU through WebAssembly. Good for internal tools and kiosks.
In a phone app
The same model files packaged inside an iOS or Android app, running on the phone's own chip, and working with no signal.
On a tablet
Field inspections, clinics, point of sale and kiosks, where the device has to keep working offline.
On an IoT device
A small ARM Linux board beside the cameras, the machines or the gate, deciding locally and sending only results.
CPU or GPU
How big a machine
does a model need?
Most of the answer is the size of the model and the precision its weights are stored at. Quantising to 4 bits cuts memory to a quarter of 16 bit for a small loss in quality, which is why small models fit on a CPU, a phone or a small edge device.
4.0 GB of weights
Fits on a laptop GPU, or a CPU with 16 GB of memory.
Parameters times bits, divided by eight. The weights only: the conversation and the working memory need more on top, and a longer context needs more again.
RAG and vector search
Answers from your documents,
with the source attached.
A model does not know your contracts or your manuals. Retrieval augmented generation finds the right passages first, by meaning rather than by keyword, and has the model answer from them. The RAG tool in the playground does every step on your device.
Air gapped deployments
When nothing may
leave the network.
An air gapped system has no path to the internet, so everything that usually arrives by download has to arrive another way. We plan for that from the first day, together with the network design our connectivity team builds.
- 01Model weights, libraries and updates arrive on approved media and are checked against signatures before install.
- 02No telemetry, no licence checks that phone home, and no package downloads at run time.
- 03Logs, prompts and outputs stay on site, with retention set by your policy.
- 04Access by role, so each person reaches only the documents their job allows.
- 05A documented way to bring in a new model version and to roll it back.
- 06A written description of where data lives and who can reach it, for your assessor.
Bring us the workload.
We will size the machine.
Tell us what the model has to do, how many people use it and what may never leave the building. We recommend the models and the hardware, install them and connect them to your files and your applications.
Discuss an edge deployment