AI Engineer Europe 2026
Gemini Nano on device — Florina Muntenescu & Oli Gaymond, Google DeepMind
Loading slides…
Read the talk
Gemini Nano on device
Android offers local, hybrid, and cloud inference, but building a useful feature also means understanding who manages the model, where app permissions apply, and which devices can run it.
From a talk by Florina Muntenescu and Oli Gaymond
Before you start: Familiarity with Android applications, model prompts, and the distinction between local and server inference will help; the Kotlin example uses only string composition.
Where should an Android AI feature run?
Where should an intelligent Android feature run—and how much infrastructure should its developer manage? Florina Muntenescu introduces the developer perspective: intelligent experiences and developer productivity. Oli Gaymond, working on Android AI product management, describes a platform spanning screen-brightness and system-memory optimization, developer infrastructure, and hardware acceleration with silicon partners. The inference APIs sit within that larger system.
The first choice is where to process a request.
| Approach | Where inference runs |
|---|---|
| On-device | On the phone |
| Hybrid | Locally when available; otherwise in the cloud |
| Cloud | On a remote service |
These are deployment choices for an application feature, not three different kinds of user interface.
Consider a feature handling banking information. With local inference, the prompt is processed on the phone without sending that information to a server. The same feature can work offline, and local calls avoid additional server inference charges. That does not eliminate the phone’s battery or storage costs. Personalization and shorter-context tasks, including translation, are other examples Muntenescu gives for local processing.
Android offers two paths into this work. ML Kit’s GenAI APIs expose Gemini Nano through a managed interface. For your own models or more customization, LiteRT-LM, called LiteRT LLM in the session, gives you more control over deployment. The distinction is how much of the model infrastructure you want to own.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
AICore makes the model a system service
Muntenescu connects Nano’s architecture to Gemma 4. That connection applies to the next generation, Gemini Nano 4, described in Google’s Gemma 4 preview announcement, rather than every existing Nano version. Android delivers the model through AICore, a system service, so applications can share a model instead of each distributing their own copy.
Gaymond compares AICore to a managed cloud service: you provide the feature and its prompt; the service handles getting the model onto the device and running it. AICore selects optimizations for the hardware available at runtime. The intended benefit is lower latency and faster execution without requiring each application developer to configure the inference stack.
Sharing the model does not mean sharing request context. Requests run in isolation. The non-retention guarantee concerns AICore’s processing: it does not retain inputs and outputs after processing a request. It does not prevent the calling application from storing data itself.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
From task APIs to hybrid inference
ML Kit already includes vision and natural-language capabilities. Its GenAI interfaces add specialized operations such as summarization, proofreading, and rewriting. The more general Prompt API accepts a natural-language request, with text and image inputs and text output. That supports image understanding, content assistance, content analysis, and entity extraction without requiring a separate task API for every feature.
Device availability creates the next constraint. Muntenescu cites Pixel 9- and Pixel 10-generation hardware, along with devices from other manufacturers, as the class supporting Nano. A feature intended for a wider audience needs another inference path when the local model is unavailable.
Firebase AI Logic’s Android hybrid inference supplies that routing: use Nano when it is available locally, otherwise run in the cloud. Muntenescu says the capability had launched a couple of weeks before the session. She points tentatively to next-generation Nano access through the AICore Developer Preview, anticipating a similar experience between local Nano 4 and cloud Gemini Flash; she does not present a quality comparison. Firebase AI Logic also provides cloud-only access to Gemini Pro and Gemini Flash-Lite, with the Gemini Developer API and Vertex AI as providers.
The platform goal is to make these paths easier to combine: local inference for privacy and responsive interactions, cloud inference for more powerful models, and a hybrid path between them. Gaymond describes API consistency as ongoing work, so developers can blend the capabilities their feature needs.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Battery cost depends on the workload
The first audience question asks about RAM and battery consumption for Nano and LiteRT models. Gaymond explains that shrinking models while preserving capability still leaves them dependent on flagship hardware. AICore exists partly to optimize that execution, but continuous inference can drain the battery quickly.
Gaymond says that intermittent question-answering or data manipulation used 10–20 times per day is not concerning for battery life. This is a workload observation, with no specified device, token workload, measured energy use, or battery percentage. It should not be read as a battery budget for a different application.
For latency-insensitive batches, he describes processing in the background while the phone charges overnight. AICore’s purpose is to absorb platform optimization work; choosing a custom model shifts more of that work back to the developer, who must profile its resource impact using the available tools. The discussion supplies no quantified runtime-RAM answer.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
The model API is below the assistant
Another attendee asks about the default assistant on an Asus phone running Android 12: when they ask for the temperature outside, does the answer come from the phone or a server? Muntenescu cannot establish which app or inference path is involved and says only that she expects it to run remotely. A familiar voice interface does not, by itself, identify where inference happens.
The follow-up asks how to install the local capability and add skills to improve incorrect answers. Muntenescu separates two user journeys: using the Gemini application and building an application that calls a model. These APIs serve the latter. An application calling an on-device model performs that inference locally; it is not extending the separate Gemini app.
Gaymond places skills one layer higher. A skill can be instructions composed into a prompt alongside the user’s query. He mentions Pocket Claw or OpenClaw as hypothetical software that could assemble those instructions and call the API, rather than as integrations demonstrated in the session.
For example, an application could compose a note-formatting instruction and a user’s text before submitting the resulting prompt. In Kotlin, that composition can remain independent of the inference client:
kotlin
fun composePrompt(instructions: String, query: String): String = """
Instructions:
$instructions
User request:
$query
""".trimIndent()
fun main() {
val prompt = composePrompt(
instructions = "Turn the supplied notes into a concise checklist. Preserve names and dates.",
query = "Call Maya on Friday. Send the draft to Leon on Monday."
)
println(prompt)
}
This prepares a request; it does not execute inference or install a skill into Nano. The higher-level application owns the instructions and workflow, while the API supplies the lower-level model capability.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Model access does not grant file access
The next question compares Nano with models in Google AI Edge Gallery, then asks whether these APIs can access pictures stored in system folders. Gaymond starts with the model boundary: Nano remains contained and configured within AICore, and applications access it through APIs. The Prompt API can support a retrieval-augmented generation workflow; he also points to a forthcoming embedding API as a way to make retrieval easier.
Gallery and AICore serve different purposes. Gaymond describes Gallery as a showcase of what local models can do, potentially using AICore backing or custom models. Exploring those capabilities can require more integration work. AICore emphasizes managed deployment and execution so a production application can focus on its prompt and feature.
The attendee makes the file question concrete: take the pictures captured during the day and turn them into a combined note. The Prompt API’s text-and-image input can support that kind of request, but the application must supply the information. It can pass files it already has permission to access; the inference API does not grant unrestricted access to the phone’s filesystem. Constructing the prompt and feeding it accessible content follows the same basic pattern as server-based inference.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Prompting and embeddings are separate capabilities
An attendee asks specifically about vectorizing text notes to find similarities between them. Gaymond’s answer is explicit: an embedding API is not yet available through this interface. He says it will arrive soon and identifies a Gemma embedding model as the intended capability. That is a forecast, not an available API in the session. Being able to place retrieved text into a prompt does not mean the same interface can already generate vectors for retrieval.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Broader device reach means owning more testing
The final question brings the discussion back to the device-and-model matrix. Are language models and embeddings the whole on-device offering, and does every capability require a recent flagship? Gaymond distinguishes classical ML Kit models—text processing, OCR, and vision—from the larger generative models. Gaymond describes classical ML Kit models as reaching a billion-plus devices, while the GenAI APIs require fairly flagship devices from the preceding couple of years. The device-count claim is a broad reach estimate, not a compatibility guarantee for every API.
The remaining choice is who owns validation across that hardware range.
| Path | Device testing responsibility |
|---|---|
| AICore | Platform manages supported model/device combinations |
| LiteRT LLM | Developer tests chosen models on target devices |
Gaymond presents AICore availability as assurance that the managed model will run well on that device. If that supported range is too narrow, LiteRT LLM lets developers pursue a wider range, but they must test it themselves, even with supporting tools. He hands that topic to Cormac’s following session. A managed model service reduces deployment and testing work; broader custom-model reach requires taking that work back.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Overview of on-device APIs, supported devices, model sharing and application quotas.
Instructions for enrolling a supported device and testing preview models and prompts.
Setup and capability limits for experimental inference using local models and cloud fallback.
Google's open-source framework for deploying language models on edge devices.
An application and source repository for exploring locally running generative AI models.
Further reading
The April 2026 announcement connecting Gemma 4 previews with the forthcoming Gemini Nano 4 generation.
Explains how EmbeddingGemma produces text representations for search and retrieval.
Full transcript
Loading transcript…