Add an AI Assistant to Your Mobile App: Cloud vs On-Device

Use a cloud API when your assistant needs strong reasoning, long context, or fresh knowledge. Use on-device AI when it must work offline, keep data on the phone, or avoid per-request fees. A hybrid design, with a small on-device model for quick tasks and a cloud model for hard ones, is often the best fit. Here is how to choose.

Not sure which path fits your app? Groooh builds React Native apps and agentic AI systems. Book a free 30 minute architecture review and we will tell you which option fits your users, budget and timeline.

What is the real difference between a cloud API and an on-device AI assistant?

A cloud API assistant sends each request over the internet to a hosted model and returns the answer. An on-device assistant runs a smaller model on the phone itself, so no request leaves the device. Cloud gives you the strongest models and instant updates. On-device gives you offline use, privacy by design, and no per-request fee.

On iPhone, Apple's Foundation Models framework gives apps access to the model behind Apple Intelligence, with offline use and no inference cost (MacRumors, on Apple's announcement). Apple describes that model as roughly 3 billion parameters (Implicator). On Android, Gemini Nano runs inside the AICore system service and is reached through ML Kit's GenAI APIs (Android Developers).

In React Native you have a third option: bundle an open model and run it with a library such as react-native-executorch. That gives you the same model on both platforms, but you ship and manage the model yourself.

Cloud API vs on-device: how do they compare?

Cloud wins on model quality, context length, and update speed. On-device wins on offline use, data privacy, and marginal cost. A gap that is easy to overlook is device reach: Apple's on-device model needs an Apple Intelligence-compatible device, and Android's depends on supported hardware. The table compares the options on the same attributes.

Sources: Claude pricing, Apple TN3193, Apple Intelligence device list (iClarified, June 9, 2026), Apple Developer Forums, Apple engineer answer, Android Gemini Nano docs, React Native ExecuTorch.

Which one should I choose for my app: cloud or on-device?

Choose cloud if your assistant must reason over long documents, answer with current knowledge, or work on every device. Choose on-device if the feature is short, private, or must work offline, such as rewriting text, extracting fields, or classifying input. If you need both, build a hybrid. Decide per feature, not per app.

Sources: Android Developers, Gemini Nano, Android Developers, Prompt API, Implicator, Apple Foundation Models update, Apple TN3193.

How much does a cloud AI assistant cost to run?

On-device inference has no per-request fee, but a cloud assistant can still be cheap. In our estimate, 10,000 monthly users sending 20 messages each cost about $60 a month on Claude Haiku 5.5 or $1,200 on Claude Sonnet 5.5. Cost scales with users, so model choice and prompt length matter most at small scale.

Our assumptions (an estimate, not a quote): 10,000 monthly active users, 20 assistant messages per user per month, 1,500 input tokens and 300 output tokens per message. That is 200,000 messages, 300 million input tokens and 60 million output tokens a month. Prices are Anthropic's list prices per million tokens (MTok), as of November 11, 2026.

The arithmetic: Haiku 5.5 is 300 MTok x $0.10 plus 60 MTok x $0.50, which is $30 plus $30. Sonnet 5.5 is 300 MTok x $2 plus 60 MTok x $10, which is $600 plus $600. The 100,000 user column is the same inputs times ten.

Three things this estimate leaves out. First, engineering and testing time, which is real for any on-device path. Second, Haiku 5.5 charges higher prices ($0.50 input, $2.50 output) when a prompt exceeds 100,000 tokens, which a chat assistant rarely reaches. Third, token counts: Claude 4.7 and later models use a tokenizer that produces about 30 percent more tokens for the same text, so measure your own prompts with the token counting endpoint.

To lower the bill, shorten the system prompt, cap output length, and use prompt caching. A cache hit costs 5 percent of the input price on Sonnet 5.5, which is $0.10 per MTok. We used Claude prices because we could verify them on the vendor's page. Other providers offer comparable tiers, so price them with your own token counts before you decide.

Sources: Claude API pricing, Anthropic, MacRumors, on Apple's Foundation Models announcement, Android Developers, Gemini Nano.

Should I use a hybrid approach?

Often yes. Run short, private, or offline tasks on the device and send hard requests to a cloud model, which also acts as the fallback for phones that cannot run the local model. Google documents this pattern for Android through Firebase AI Logic, and Apple's framework can now route work to Private Cloud Compute or a server model.

Google's guidance is to route to Gemini Flash or Pro in the cloud when Gemini Nano is unavailable, and to protect those cloud calls with Firebase App Check (Android Developers). Apple's updated documentation describes routing to on-device models, Private Cloud Compute, or another server model provider when a task needs more capacity (Implicator).

Want this routing designed for your app? We map each feature to cloud, on-device or hybrid, then price it. Book a free 30 minute architecture review.

Can a React Native app run an on-device AI model?

Yes. Libraries such as react-native-executorch, which wraps Meta's ExecuTorch, run LLMs, vision, and speech models on the phone from JavaScript. It needs iOS 17 or Android 13 and React Native's New Architecture. Apple's Foundation Models and Android's ML Kit are native APIs, so you need a native module or a wrapper library.

The Prompt API was announced as an Alpha on October 30, 2025 (Android Developers Blog). Check its current status before you build on it, since this post could not confirm whether it has since left alpha.

One product consequence: the same question can get different answers on an iPhone, a Pixel and a budget phone. Apple says there is no model pinning or version API, and recommends its Evaluations framework to catch regressions after OS updates (Apple Developer Forums). Build a small test set of your real prompts and run it on every OS release.

What is the step-by-step plan to add an AI assistant to my app?

Start with one job the assistant must do, pick the path for each feature, build a thin prototype, test it on real devices, and add a fallback before launch. A working version of a single feature beats a general chatbot, because you can measure it and fix it. Use the steps below as a checklist.

1. Name one job. Write the single task the assistant must do, such as "summarize a support thread" or "answer questions about my order". Skip a general chatbot for version one.
2. Sort each feature by the table above. Mark it cloud, on-device or hybrid using length, privacy and offline needs.
3. Check your device mix. Look at your analytics for the share of users on Apple Intelligence-compatible iPhones and on Android phones that support Gemini Nano. This number decides how much of your audience gets the local path.
4. Prototype the cloud path first. Call a model through your own backend, never from the app with a key inside it. Log real user questions for two to four weeks.
5. Move the right requests on-device. Pick short, frequent or private requests from the logs and test them with Apple's Foundation Models, ML Kit or react-native-executorch.
6. Add a fallback and a failure message. Handle a missing local model, a full context window, and no network.
7. Build a test set of 30 to 50 real prompts. Run it on every OS release and model change, because Apple offers no model pinning.
8. Set cost alerts and rate limits. Cap tokens per user per day so one heavy user cannot inflate the bill.
9. Launch to a small group, then widen. Use a feature flag so you can switch paths without an app store release.

How does Groooh approach this?

Disclosure: Groooh sells React Native development and agentic AI services, so we have a commercial interest in this topic. The comparison above is built from vendor documentation so you can check it yourself.

Groooh is a design, development and agentic AI agency that builds React Native mobile apps and agentic AI systems with LangChain, LangGraph and RAG pipelines for clients in the US, UK and Europe. For an AI assistant, our default is to start with a cloud prototype behind your own backend, learn from real usage, and then move only the right requests on-device.

Have an assistant feature in mind? Book a free 30 minute architecture review and leave with a recommended path, a rough monthly cost, and a list of risks for your app.

When is this advice wrong?

This advice breaks down if most of your users are on older phones, if the assistant must read long documents, or if a contract limits where data can go. Each condition flips the default, so check it against your real users and contracts before you commit.

The best case for the opposite view: on-device costs nothing per request at any scale, and cloud bills grow with every user. That is true, and it is why heavy, repetitive features belong on the device. But it does not remove the cost of building, testing and maintaining native paths on two platforms.

FAQ and follow-up questions

Which one should I choose, a cloud API or on-device AI?

Choose per feature. Use a cloud API for open-ended reasoning, long context, and current knowledge. Use on-device AI for short, private, or offline tasks. If your app has both kinds of features, build a hybrid with the cloud as the fallback. Pure on-device is a poor fit when you must support older phones.

Is on-device AI more private than a cloud API?

On-device is stronger by design, because requests never leave the phone, and both Apple and Google position their local models that way. A cloud assistant can still be private enough if you send minimal data, disclose it clearly, and pick a provider whose terms fit your compliance needs. Ask counsel about regulated data.

How much does an AI assistant cost per month?

In our estimate, 10,000 monthly users sending 20 messages each costs about $60 a month on Claude Haiku 5.5 or $1,200 on Claude Sonnet 5.5, using the vendor's published prices. Your real cost depends on prompt length and traffic. On-device inference has no per-request fee, but you pay in engineering and testing.

Can I run an on-device LLM in React Native?

Yes. React Native ExecuTorch runs LLMs, vision, and speech models on the phone from JavaScript. It supports iOS 17 and Android 13 or newer and requires the New Architecture. Apple's Foundation Models and Android's ML Kit are native APIs, so you need a native module or wrapper library to use them.

Will on-device AI work on every phone?

No. Apple's on-device model needs an Apple Intelligence-compatible device, which starts at the iPhone 15 Pro. Android's Gemini Nano depends on device support and AICore, and Google's Prompt API performed best on Pixel 10 phones at its alpha launch. Plan a cloud fallback for everyone else.

Should I start with cloud or on-device?

Start with the cloud. It is one integration, works on every device, and shows you what users actually ask. Once you see which requests are short, private, or frequent, move those on-device and keep the cloud for the rest. This avoids building a local model path before you know you need it.

How do I keep my API key safe in a mobile app?

Do not ship the key inside the app. Call your own backend, which holds the key and system prompt and applies rate limits. On Android, Google's Firebase AI Logic hybrid path pairs with App Check to protect cloud calls and keep sensitive system prompts server-side. Treat any key in an app binary as public.

What should I ask an agency before hiring them to build this?

Ask which path they would choose for each feature and why, how they will test on real devices, how they handle fallback and OS updates, and what the monthly token bill looks like at your expected usage. A good answer includes numbers and a plan for Apple and Android differences, not only a demo.

Next step

The right answer depends on your users' phones, your data rules and your usage. Book a free 30 minute architecture review with Groooh and we will map each feature to cloud, on-device or hybrid, with a cost estimate.

Last updated: November 12, 2026. Disclosure: Groooh sells React Native and agentic AI development services.

Related Links: Mobile App Development Services, UIUX Design Services, Agentic AI Services

Sources

Prices and platform details as of November 11, 2026.

• Claude API pricing, Anthropic
• Gemini Nano and ML Kit GenAI APIs, Android Developers
• ML Kit Prompt API Alpha announcement, Android Developers Blog, October 30, 2025
• TN3193: Managing the on-device foundation model's context window, Apple Developer
• On-device model capabilities, limits, and versioning, Apple Developer Forums (Apple engineer answer, June 2026)
• Apple Announces Foundation Models Framework, MacRumors, June 9, 2025
• Apple expands Xcode and Foundation Models for WWDC AI developers, Implicator, June 8, 2026
• Apple Intelligence compatible devices, iClarified, June 9, 2026
• react-native-executorch, Software Mansion, GitHub