A couple of years ago, “add AI to the frontend” meant one thing: call an API on a server. Today, a surprising amount of AI can run directly in the user’s browser, on their own GPU, with no network round trip and no per-request bill.
I’ve been experimenting with this a lot, and I want to share what actually works today, which APIs to use, and the trade-offs that only show up once you ship it to real users on real devices.
Why run AI in the browser at all?
There are four real reasons. If none of them apply to you, stick with a server.
- Privacy. The text never leaves the device. That’s a big deal for notes, emails, health data, or anything a legal team will ask about.
- Latency. No round trip. For features that run on every keystroke (like classification or suggestions), that matters a lot.
- Cost. The user’s hardware does the work. A feature that runs a million times a day costs you nothing per call.
- Offline. It keeps working on a plane.
The catch: you get much smaller models than what runs... premium content.
