AI Engineer Europe 2026

Gemini Nano on device — Florina Muntenescu & Oli Gaymond, Google DeepMind

Loading slides…

Read the talk

Gemini Nano on device

Android offers local, hybrid, and cloud inference, but building a useful feature also means understanding who manages the model, where app permissions apply, and which devices can run it.

From a talk by Florina Muntenescu and Oli Gaymond

Before you start: Familiarity with Android applications, model prompts, and the distinction between local and server inference will help; the Kotlin example uses only string composition.

Where should an Android AI feature run?

Where should an intelligent Android feature run—and how much infrastructure should its developer manage? Florina Muntenescu introduces the developer perspective: intelligent experiences and developer productivity. Oli Gaymond, working on Android AI product management, describes a platform spanning screen-brightness and system-memory optimization, developer infrastructure, and hardware acceleration with silicon partners. The inference APIs sit within that larger system.

The first choice is where to process a request.

ApproachWhere inference runs
On-deviceOn the phone
HybridLocally when available; otherwise in the cloud
CloudOn a remote service

These are deployment choices for an application feature, not three different kinds of user interface.

Slide titled “AI on Android” with three rows for on-device inference, hybrid inference and cloud inference.Open full source frame
AI on Android: on-device, hybrid and cloud inference.

Consider a feature handling banking information. With local inference, the prompt is processed on the phone without sending that information to a server. The same feature can work offline, and local calls avoid additional server inference charges. That does not eliminate the phone’s battery or storage costs. Personalization and shorter-context tasks, including translation, are other examples Muntenescu gives for local processing.

Android offers two paths into this work. ML Kit’s GenAI APIs expose Gemini Nano through a managed interface. For your own models or more customization, LiteRT-LM, called LiteRT LLM in the session, gives you more control over deployment. The distinction is how much of the model infrastructure you want to own.

0:160:33
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:16 · section reference included

AICore makes the model a system service

Muntenescu connects Nano’s architecture to Gemma 4. That connection applies to the next generation, Gemini Nano 4, described in Google’s Gemma 4 preview announcement, rather than every existing Nano version. Android delivers the model through AICore, a system service, so applications can share a model instead of each distributing their own copy.

Gaymond compares AICore to a managed cloud service: you provide the feature and its prompt; the service handles getting the model onto the device and running it. AICore selects optimizations for the hardware available at runtime. The intended benefit is lower latency and faster execution without requiring each application developer to configure the inference stack.

Sharing the model does not mean sharing request context. Requests run in isolation. The non-retention guarantee concerns AICore’s processing: it does not retain inputs and outputs after processing a request. It does not prevent the calling application from storing data itself.

3:113:26
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:11 · section reference included

From task APIs to hybrid inference

ML Kit already includes vision and natural-language capabilities. Its GenAI interfaces add specialized operations such as summarization, proofreading, and rewriting. The more general Prompt API accepts a natural-language request, with text and image inputs and text output. That supports image understanding, content assistance, content analysis, and entity extraction without requiring a separate task API for every feature.

Device availability creates the next constraint. Muntenescu cites Pixel 9- and Pixel 10-generation hardware, along with devices from other manufacturers, as the class supporting Nano. A feature intended for a wider audience needs another inference path when the local model is unavailable.

Firebase AI Logic’s Android hybrid inference supplies that routing: use Nano when it is available locally, otherwise run in the cloud. Muntenescu says the capability had launched a couple of weeks before the session. She points tentatively to next-generation Nano access through the AICore Developer Preview, anticipating a similar experience between local Nano 4 and cloud Gemini Flash; she does not present a quality comparison. Firebase AI Logic also provides cloud-only access to Gemini Pro and Gemini Flash-Lite, with the Gemini Developer API and Vertex AI as providers.

The platform goal is to make these paths easier to combine: local inference for privacy and responsive interactions, cloud inference for more powerful models, and a hybrid path between them. Gaymond describes API consistency as ongoing work, so developers can blend the capabilities their feature needs.

4:495:07
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:49 · section reference included

Battery cost depends on the workload

The first audience question asks about RAM and battery consumption for Nano and LiteRT models. Gaymond explains that shrinking models while preserving capability still leaves them dependent on flagship hardware. AICore exists partly to optimize that execution, but continuous inference can drain the battery quickly.

Gaymond says that intermittent question-answering or data manipulation used 10–20 times per day is not concerning for battery life. This is a workload observation, with no specified device, token workload, measured energy use, or battery percentage. It should not be read as a battery budget for a different application.

For latency-insensitive batches, he describes processing in the background while the phone charges overnight. AICore’s purpose is to absorb platform optimization work; choosing a custom model shifts more of that work back to the developer, who must profile its resource impact using the available tools. The discussion supplies no quantified runtime-RAM answer.

8:168:25
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

8:16 · section reference included

What happens when many apps share Nano?

An audience member extends the shared-model idea to a phone with 100 applications using it. Does that create a bottleneck? Gaymond first separates the cost of delivering the model from the cost of executing requests. Gaymond estimates the smallest useful models at roughly 1 GB and the models being shipped at roughly 3–4 GB in total. Those are model-size estimates, not measurements of runtime RAM. Packaging that much data independently is a substantial burden for an app; a system copy lets many apps share the distribution and storage cost.

Execution still consumes resources. Central management lets AICore queue work and attribute battery consumption to the applications using it. Gaymond sketches a hypothetical question about whether the user wants an app to consume so much quota, then clarifies that the system does not actually show that prompt. His comparison is GPS or Wi-Fi: users may accept the battery cost when a feature is valuable, and reject it when it is not.

AICore handles scheduling rather than requiring each developer to coordinate with other applications. In the session’s explanation, background requests queue while the actively used foreground app receives top priority. For implementation today, the documented boundary is stricter: the ML Kit documentation checked in August 2026 permits GenAI inference only from the top foreground app; background calls, including those from foreground services, receive BACKGROUND_USE_BLOCKED. The earlier overnight-processing discussion is therefore not an implementation recipe for that current API.

10:1310:25
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

10:13 · section reference included

The model API is below the assistant

Another attendee asks about the default assistant on an Asus phone running Android 12: when they ask for the temperature outside, does the answer come from the phone or a server? Muntenescu cannot establish which app or inference path is involved and says only that she expects it to run remotely. A familiar voice interface does not, by itself, identify where inference happens.

The follow-up asks how to install the local capability and add skills to improve incorrect answers. Muntenescu separates two user journeys: using the Gemini application and building an application that calls a model. These APIs serve the latter. An application calling an on-device model performs that inference locally; it is not extending the separate Gemini app.

Gaymond places skills one layer higher. A skill can be instructions composed into a prompt alongside the user’s query. He mentions Pocket Claw or OpenClaw as hypothetical software that could assemble those instructions and call the API, rather than as integrations demonstrated in the session.

For example, an application could compose a note-formatting instruction and a user’s text before submitting the resulting prompt. In Kotlin, that composition can remain independent of the inference client:

kotlin

fun composePrompt(instructions: String, query: String): String = """
    Instructions:
    $instructions

    User request:
    $query
""".trimIndent()

fun main() {
    val prompt = composePrompt(
        instructions = "Turn the supplied notes into a concise checklist. Preserve names and dates.",
        query = "Call Maya on Friday. Send the draft to Leon on Monday."
    )
    println(prompt)
}

This prepares a request; it does not execute inference or install a skill into Nano. The higher-level application owns the instructions and workflow, while the API supplies the lower-level model capability.

12:4312:52
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

12:43 · section reference included

Prompting and embeddings are separate capabilities

An attendee asks specifically about vectorizing text notes to find similarities between them. Gaymond’s answer is explicit: an embedding API is not yet available through this interface. He says it will arrive soon and identifies a Gemma embedding model as the intended capability. That is a forecast, not an available API in the session. Being able to place retrieved text into a prompt does not mean the same interface can already generate vectors for retrieval.

17:2317:33
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

17:23 · section reference included

Broader device reach means owning more testing

The final question brings the discussion back to the device-and-model matrix. Are language models and embeddings the whole on-device offering, and does every capability require a recent flagship? Gaymond distinguishes classical ML Kit models—text processing, OCR, and vision—from the larger generative models. Gaymond describes classical ML Kit models as reaching a billion-plus devices, while the GenAI APIs require fairly flagship devices from the preceding couple of years. The device-count claim is a broad reach estimate, not a compatibility guarantee for every API.

The remaining choice is who owns validation across that hardware range.

PathDevice testing responsibility
AICorePlatform manages supported model/device combinations
LiteRT LLMDeveloper tests chosen models on target devices

Gaymond presents AICore availability as assurance that the managed model will run well on that device. If that supported range is too narrow, LiteRT LLM lets developers pursue a wider range, but they must test it themselves, even with supporting tools. He hands that topic to Cormac’s following session. A managed model service reduces deployment and testing work; broader custom-model reach requires taking that work back.

17:5118:04
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

17:51 · section reference included

Resources

From the talk

Transcript download
Full transcript

Loading transcript…