Your AI Bill Shouldn't Grow With Your Success

There is a question most teams ask only after an AI feature succeeds: what does each request actually cost us?

Cloud AI pricing looks harmless per token. It stops looking harmless when a feature works. Every improvement in engagement makes the bill bigger - your best-case product scenario and your worst-case infrastructure scenario are the same scenario. A meter that never stops running is a strange thing to attach to the feature you most want people to use.

In 2026 there is a real alternative for a large slice of that traffic, and it changes how the unit economics of AI features work.

The math nobody does before shipping

Take an unremarkable, popular feature: short summarization - notifications, messages, documents, reviews. Say it does 1M requests a day at 500 tokens in, 100 tokens out.

On 2026 cloud pricing that workload costs roughly $8,250 per month on a budget model (Gemini 3.1 Flash-Lite at $0.25/$1.50 per 1M tokens) and about $16,500 per month on a mid-tier one (Gemini 3 Flash). Step up the model class and the multiplier follows: GPT-5.4 Mini runs $0.75/$4.50 per 1M tokens, GPT-5.4 $2.50/$15, frontier models around $5/$30.

That is one feature. Ship three AI features and grow 5x, and the line item that started as "negligible" is a six-figure annual commitment that scales linearly with your success - plus it breaks offline, adds a network round-trip to every interaction, and sends user data to a third party along the way.

The same workload, side by side:

What changed in 2026

Both platform vendors now ship a production-grade LLM inside the operating system, callable by any app, at no per-request cost.

• Apple: the Foundation Models framework gives every iOS app a free API to an on-device ~3B model. Apple's own phrase is "zero token costs." On top of that, apps in the App Store Small Business Program (under 2M lifetime downloads) get Apple's Private Cloud Compute model - the cloud tier - at zero API cost too.
• Google: Gemini Nano runs on more than 140M Android devices via AICore, with "no additional per-use charges." The ML Kit GenAI APIs (summarization, rewriting, image description, general prompting) are GA.

The economic structure of this is the point: on-device AI converts a variable operating cost into a mostly fixed engineering cost. You pay once - in development - and then ten users or ten million cost the same to serve. Cloud AI is a meter; on-device is buying the coffee machine instead of paying per cup.

How much of a real workload can actually move?

The fair objection: surely the free on-device models are too small for real work. The measured answer is more interesting than the intuition.

TrafficBench, a study of standard enterprise LLM traffic, found that 80.7% of queries are executable on-device with models under 20B parameters. Most production AI traffic is not deep reasoning - it is summarize this, classify that, extract these fields, rewrite this sentence. Short-context, high-frequency, well within device-scale capability.

The same study measured what a hybrid router - on-device first, cloud for the hard cases - does to the bill: total cost down 60%, GPU compute down 67%, energy down 77%, with under 5% quality loss. That is the shape of the opportunity: not "replace the cloud," but "stop paying cloud prices for the 80% of traffic that never needed a frontier model."

The company that ran the test

Kakao Mobility benchmarked cloud Gemini against on-device Gemini Nano in production - same feature, head-to-head - and chose on-device on privacy, cost, accuracy, and speed together. The business results, not the benchmark results: order completion 24% faster, new-user conversion up 45%, and more than 200% growth in AI-assisted orders during peak seasons.

Worth noting what happened there: the cheaper architecture was also the faster one, because a model on the phone answers with zero network round-trip. Cost and latency usually trade off against each other. On-device, for fitting workloads, they improve together.

What this doesn't mean

Honesty section, because this is where vendor blog posts oversell. Not everything moves on-device, and part of our job is telling clients what doesn't:

• The hard 20% stays cloud-side. Deep multi-step reasoning, long documents (system models cap at 4K tokens of context on most devices), broad world knowledge, and frontier-quality generation still need big models.
• The device fleet is uneven. The best on-device models want 12GB of RAM - flagship territory. Your architecture needs a capability check and a cloud fallback for older devices.
• Physics applies. Phones throttle under sustained load; on-device is built for frequent short bursts, not hour-long batch jobs.

We covered the full technical picture - platforms, benchmarks, thermal limits - in the state of the union post. The summary for a business audience: the constraint set is real, well-mapped, and exactly why the winning architecture is hybrid. Everyday tasks run on the phone; heavy exceptions go to the cloud. The skill is knowing where to draw the line.

Three questions to ask about your own product

If you want to know whether this applies to you, the screen is three questions per AI feature:

1. How often does it fire? High-frequency features (every message, every photo, every session) benefit most - that is where variable cost compounds and where $0 marginal changes the P&L.
2. How much context does it need? If a request fits in a few thousand tokens - most summarize/classify/extract/rewrite features do - it is a candidate.
3. How sensitive is the data? Photos, messages, health metrics, documents, location. Processing on the device means the data never leaves it - which turns a privacy liability into a selling point, and makes conversations with your lawyers noticeably simpler.

Features that score high on all three are your on-device layer. What remains is your cloud layer - now a much smaller bill, spent only where big models earn it.

Drawing that line for a specific product - which journeys move now, what the fallback architecture looks like, what it costs to build - is what we do. The first conversation is free and ends with a written report you keep either way.