# Sovereign Escape Velocity: Ownership with Open Models — Gus Martins and Ian Ballantyne, Google DeepMind

Gus Martins · Ian Ballantyne

AI Engineer Europe 2026

[Talk page](https://ai.engineer/talks/SS-A8sE7hkw-sovereign-escape-velocity-ownership-open-models) · [Video](https://www.youtube.com/watch?v=SS-A8sE7hkw)

## Sovereign Escape Velocity: Ownership with Open Models

Owning a model changes where data can go, which tasks are economical, and who operates the system. Gemma 4 illustrates those choices from phone skills to local translation agents.

Familiarity with model inference, GPU memory, and API clients will help with the deployment discussion and Python example.

### When proprietary data cannot leave

[1:31](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=91s)

What happens when a model needs access to proprietary data that cannot leave your infrastructure? A hosted API may offer excellent capabilities, but it cannot satisfy that constraint by intelligence alone. You need control over where inference runs, access to the model itself, and the ability to customize it.

That is the division of responsibility between **Gemini** and **[Gemma 4](https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/)** in Gus Martins and Ian Ballantyne’s Google DeepMind presentation. Martins, who works on Gemma, introduces the family released the previous Thursday, with further launch detail reserved for Omar’s keynote and Cassidy’s session. He presents Gemini as Google’s strongest, easiest starting point; Gemma supplies the control that a model accessed through Google’s hosted API cannot. The two families serve complementary deployment needs.

### What the effective sizes mean

[2:18](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=138s)

The launch comprises four sizes. The smaller **Gemma 4 E2B** and **Gemma 4 E4B** target phones, IoT hardware, and other constrained devices. The `E` means **effective**: the name describes the effective parameter scale, rather than counting every parameter associated with the model.

Martins explains E2B as roughly a 5B-parameter model whose transformer computation has a much smaller footprint. Additional token-mapping parameters can reside outside accelerator memory. The [model card](https://ai.google.dev/gemma/docs/core/model_card_4) describes these as per-layer embedding lookups. This is the useful distinction behind the naming: **effective parameter counts are not gigabytes of memory**. Actual memory requirements also depend on representation and runtime configuration; neither `E2B` nor `E4B` is a direct GPU-memory specification.

That arrangement is intended to make local execution practical on a Pixel or another phone. Both small models accept text, vision, and audio input, but produce text output. They also support thinking, coding, and function calling, so the phone deployment is not limited to a plain text chatbot.

### Active computation, total weights, and useful capability

[3:42](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=222s)

The larger options are **Gemma 4 26B A4B**, a mixture of experts, and **Gemma 4 31B Dense**. In the mixture-of-experts model, only part of the model participates in each token’s computation. Martins describes this through roughly 4B-sized working components. That figure refers to active computation, not the space needed to store the full model. The dense 31B is presented as the strongest model in the family.

Martins estimates that the two larger models stood fourth and seventh among open models on LM Arena at talk time, without assigning a rank to each. He also describes other models in the top twenty or thirty as at least two or three times larger, and sometimes twenty times larger. These are his leaderboard snapshot and size comparison, rather than a controlled task evaluation. The practical attraction is capability at a size that can fit more deployment environments; he cites coding, agentic work, and multilingual use as regular applications of the 31B model.

### Enough capability for the task

[4:59](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=299s)

Both larger models can run on a desktop or behind a server endpoint. Hosting them in a cloud environment still leaves room for ownership: the relevant question is who controls the model and its deployment, not whether the hardware sits under someone’s desk.

![Slide labeled “26B A4B” and “31B Dense,” with a blue cloud icon surrounded by a dotted cloud outline.](https://ai.engineer/api/talk-essay-media/5e4808335c253df1f1bb43cf29e4de59487cb3531b858ceda15902fe1242d125)

26B A4B and 31B Dense models shown with a cloud graphic.

**Choose for the work that needs doing.** Martins explicitly concedes that these are not the most intelligent models available. Email summarization, coding assistance, document search, and interactions with documents do not necessarily require that distinction. A sufficiently capable smaller model can make those workloads accessible on less hardware.

Martins contrasts a 31B deployment on one GPU with unnamed competitors requiring 200 GB of memory and perhaps four or five GPUs. The comparison does not specify workload or configuration. The launch announcement supplies a narrower hardware condition: unquantized bfloat16 weights fit on an 80 GB H100. Fitting weights is a starting point for deployment sizing, not a measurement of serving throughput.

For an initial trial, Martins points to `ai.dev`, now redirecting to [Google AI Studio](https://aistudio.google.com/welcome). At the time of the talk, he describes free access to the 26B and 31B models alongside Gemini and **Veo**. He mentions combining vision, thinking, and code execution, although his planned demonstration of that combination is not shown.

The emphasis on LM Arena also has a specific purpose: its Elo scores reflect human preferences about responses. Academic benchmarks help characterize capabilities, while preference evaluation addresses how a model feels to query and how customers may experience its answers. Those are related but different questions.

### Licensing is part of operational independence

[7:03](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=423s)

Ownership also means being able to adapt a model and continue operating it without depending on a provider’s continued service or permission to access an endpoint. That is the operational meaning of sovereignty here: an institution can retain and run the model it has adopted.

Through **Gemma 3**, the family used a custom Gemma license that Martins describes as commercially friendly. But a custom license still requires interpretation. He illustrates the institutional friction with a hypothetical eighteen-month procurement review: even favorable terms can be difficult to adopt when legal teams must evaluate unfamiliar language. Gemma 4 moves to **Apache 2.0**, which Martins also announces as the direction for subsequent releases. That change does not retroactively change the licenses of earlier Gemma models.

The national examples span several generations. Martins mentions Gemma being used in parts of Ukraine’s services, a Bulgarian-language model based on **Gemma 2**, and work toward bringing that Bulgarian effort onto Gemma 4. He then describes a Brazilian adaptation of Gemma 3 for Portuguese. The Ukrainian implementation is not specified, and the language adaptations are not named in the talk.

Those adaptations raise a changing fine-tuning question. If the base model already performs strongly in a language, tooling may no longer be the main obstacle; finding a worthwhile improvement can be harder. Martins illustrates the tradeoff as substantial work for perhaps a one-percent gain, without specifying a metric. He reports top-two-or-three language rankings for the 31B model on LM Arena in many languages, but does not identify the languages. The decision is therefore whether a particular adaptation improves the institution’s tasks enough to justify the effort.

### Four thresholds for offloading agent work

[9:38](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=578s)

Ballantyne turns from model size to the amount of work agents generate. Agentic tasks consume tokens through repeated instructions, intermediate outputs, and subsequent actions. Taking ownership of inference can make that workload more controllable, especially when suitable hardware is already paid for. Citing [OpenRouter’s December 2025 State of AI report](https://openrouter.ai/state-of-ai), he identifies programming as one of the high-token categories when input and output are considered together. The report describes traffic through OpenRouter, rather than all AI usage.

A laptop GPU can potentially absorb document processing, data analysis, research, or suitable coding tasks. Ballantyne draws a boundary around that last category: he would not ask this model to redesign an entire application architecture. He favors specific instructions for refactoring, analysis, and generating small modular pieces of code. Those bounded jobs provide a more tractable unit of work to move onto personal hardware or a single GPU.

| Threshold | Question to answer |
|---|---|
| Capability | Can the model complete this specific task? |
| Hardware fit | Can the intended machine run it? |
| Latency | Can it respond within the task’s time budget? |
| Cost | Does owned or leased compute make economic sense? |

These thresholds have to hold together. Ballantyne describes user-facing work as potentially needing a response within a couple of seconds, whereas batch processing can tolerate a different schedule. Already-owned infrastructure, a planned hardware purchase, and leased GPU time each produce a different calculation. The useful question is which tasks you can fully offload and operate yourself.

### From phone input to a defined action

[12:45](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=765s)

The mobile demonstration, credited to Cormac’s team, puts agent skills on a phone. A user can speak, provide images, show the surrounding world, or chat with the model. The model then consults the available skills: those can trigger device applications such as calendars and maps, or invoke skills defined by the developer. The mechanism connects multimodal input to a bounded set of actions the application exposes.

![“Personal & Edge” slide with a phone chat showing a skill-call entry and a populated response, beside notes on local hardware, desktop GPUs, and battery use.](https://ai.engineer/api/talk-essay-media/bfa5eeac80374950aa0a19236b3e94021c3a54399de7d12fea0b51eb5e93f310)

A phone agent displays a response alongside personal and edge deployment considerations.

Ballantyne identifies reasoning about which action to take, followed by reliable calls to defined functions, as an improvement over the previous generation. The playground is [Google AI Edge Gallery](https://github.com/google-ai-edge/gallery), available on iOS and Android. He tentatively identifies the demonstration as using the smaller 2B option; a 4B option is available for hardware with sufficient capacity.

### Local inference changes the operating budget

[13:56](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=836s)

Moving up to desktops and single GPUs brings the 26B and 31B models into consideration. But removing a per-token bill does not remove inference cost. GPU and NPU utilization consumes energy, so execution timing becomes a product decision. A response to a photograph may need to arrive immediately; other processing can run offline in the background while a phone charges overnight. **Ownership makes scheduling and device utilization part of the application design.**

For enterprise deployments, Ballantyne contrasts earlier choices involving models above 300B parameters and multiple GPUs with possible deployments on a single H100 or A100, and an L4 in some cases. Serving a team or company still requires a calculation for the actual workload: the accelerator name alone does not determine capacity or savings.

Customization adds another reason to operate the model yourself. Ballantyne introduces **[MedGemma](https://developers.google.com/health-ai-developer-foundations/medgemma)** as a medical model family that can work with private data under an organization’s control. He offers a hospital deployment on one or two GPUs as an illustrative possibility, not measured hospital-wide serving capacity. The enterprise opportunity combines specialization with control over where sensitive data is processed.

### A local orchestrator fans out translation work

[16:08](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=968s)

The desktop demonstration uses [LM Studio](https://lmstudio.ai/docs/developer/openai-compat) to run the 26B model, the faster of the two larger options, with approximately 4B activated parameters. Ballantyne reports approximately 26 GB of RAM usage, including context, on an M4 Mac with approximately 48 GB of unified memory. This is a concrete reminder that active parameter count and total runtime memory are different quantities.

The workflow starts in a terminal:

1. An orchestrator receives the Gemma 4 announcement as the source document.
2. It distributes translation work to sub-agents, each represented by a separate window.
3. The sub-agents generate translations into different languages on the same local machine.

The initial wait includes processing in LM Studio before translations begin appearing. Multiple agent windows here represent separate pieces of work, not separate physical machines.

The intended final step is to collect the translations and generate a small webpage containing the results. Ballantyne describes that aggregation prospectively; the demonstration establishes translation generation but does not clearly establish the final webpage’s contents. He offers file processing and additional analysis as other tasks that could use the same local orchestration pattern.

### Change the endpoint, then evaluate the workflow

[18:14](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=1094s)

The first integration step is small: point an existing OpenAI-compatible client at LM Studio or [Ollama](https://docs.ollama.com/api/openai-compatibility), then select the locally served Gemma model. Current compatibility documentation supports this endpoint-and-model pattern for supported APIs; it does not imply complete feature equivalence. In Python, the translation task can start with a client like this, using the model identifier exposed by the local server:

```python
import os
from pathlib import Path

from openai import OpenAI

client = OpenAI(
    base_url=os.environ["LOCAL_OPENAI_BASE_URL"],
    api_key=os.environ.get("LOCAL_OPENAI_API_KEY", "local"),
)

announcement = Path("announcement.txt").read_text(encoding="utf-8")
response = client.chat.completions.create(
    model=os.environ["LOCAL_GEMMA_MODEL"],
    messages=[
        {
            "role": "user",
            "content": (
                "Translate this Gemma 4 announcement into Portuguese. "
                "Preserve model names and technical identifiers. "
                "Return only the translation.\n\n" + announcement
            ),
        }
    ],
)

translation = response.choices[0].message.content
if not translation:
    raise RuntimeError("The model returned no translation text.")

Path("announcement.pt.txt").write_text(translation, encoding="utf-8")
```

This makes one translation request through the local endpoint. An orchestrator can assign the same source document to separate language tasks, as in the demonstration.

The next steps are empirical:

1. **Drop the model into an existing workflow.** Observe which tasks it handles well.
2. **Separate tuning needs from capability limits.** Some failures may be addressable through adaptation; others may reflect task complexity beyond the model.
3. **Strengthen the evaluation suite for your tasks.** General benchmarks help identify candidates, but success depends on the work your application actually asks the model to perform.

The endpoint change gets the experiment started. Task evaluation determines whether the substitution is useful.

### The responsibilities that come with control

[19:20](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=1160s)

Self-hosting transfers uptime, downtime, maintenance, and ongoing operating costs to the owner. Buying hardware also adds upfront capital expenditure. Those obligations belong in the same decision as model quality and inference cost; they are part of what it means to control the service.

Mobile deployment adds a support matrix: which devices are supported, which accelerators they contain, and how much RAM is available. In exchange, the application can work offline or process private information that never leaves the user’s device. Enterprise scaling requires the same attention to infrastructure sizing and continuing operating costs at a larger scope.

![“Best Practices & Next Steps” slide with four bullets on drop-in evaluation, custom evaluation, serving management, and enterprise scaling, plus a Python code panel.](https://ai.engineer/api/talk-essay-media/074d5e95c93a3b266f651e8189c8610ea5c37de801cafc3e2c550e76af481e33)

Best practices cover evaluation, serving management, and enterprise scaling alongside a Python example.

Ballantyne closes by encouraging experimentation with concrete tasks, using benchmarks as signals of what may be feasible. The feedback he asks for is equally concrete: how deployments behave, what fine-tuning changes, and which problems emerge. Those observations are what turn access to an open model into a system an organization can confidently operate.

## Resources

- [Gemma 4 launch announcement](https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/) — The original announcement covering the four launch sizes, Apache 2.0 licensing, capabilities and hardware options.
- [Gemma 4 model card](https://ai.google.dev/gemma/docs/core/model_card_4) — Current architecture tables, parameter definitions and evaluations, including the expanded five-model family.
- [Google AI Edge Gallery](https://github.com/google-ai-edge/gallery) — Installable playground for experimenting with local mobile models, multimodal inputs and agent skills.
- [OpenRouter State of AI](https://openrouter.ai/state-of-ai) — December 2025 study of more than 100 trillion tokens, including programming workloads and input/output sequence lengths.
- [LM Studio OpenAI compatibility](https://lmstudio.ai/docs/developer/openai-compat) — Configure an existing client to use a locally served model through supported OpenAI-compatible endpoints.
- [Ollama OpenAI compatibility](https://docs.ollama.com/api/openai-compatibility) — Current documentation for connecting OpenAI-compatible applications to Ollama.
- [MedGemma](https://developers.google.com/health-ai-developer-foundations/medgemma) — Google's developer overview of the MedGemma medical model family.
- [GAIA for Brazilian Portuguese](https://deepmind.google/models/gemma/gemmaverse/gaia/) — Case study of adapting Gemma 3 4B to Brazilian Portuguese through continuous pretraining.
- [BgGPT Gemma 2 model card](https://huggingface.co/INSAIT-Institute/BgGPT-Gemma-2-9B-IT-v1.0) — INSAIT's Bulgarian-language Gemma 2 adaptation, with model details and usage instructions.

## Slides

### AI Engineer Europe

[0:00](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=0s) [AI Engineer Europe logo in white on black.](https://assets.ai.engineer/sites/aiecode2025/sha256/24e440bf5dfd4ecccd4fffdc9ef9243be07fff10b16e87c6b80fbc594828e3ea.webp)

AI Engineer
EUROPE

### Presenting Sponsor

Google DeepMind is the presenting sponsor.

[0:05](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=5s) [Presenting sponsor Google DeepMind on a black background.](https://assets.ai.engineer/sites/aiecode2025/sha256/9a43b90b83b6eca77e82f00387f025461fb2cd84d99ed7f55e0f7cc2bbdddc7d.webp)

PRESENTING SPONSOR

Google DeepMind

### Platinum Sponsors

[0:10](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=10s) [Platinum sponsors Braintrust, WorkOS, and OpenAI on a black background.](https://assets.ai.engineer/sites/aiecode2025/sha256/2d35d6078675fe9a03b15d0c159e74531fc0a3308f8622c03ad0d462180cfe3c.webp)

PLATINUM SPONSORS

Braintrust
WorkOS
OpenAI

### Gemma 4

Gus introduces Gemma 4 as an open model family for running and customizing models on your own hardware, including when proprietary data must stay within your infrastructure.

[0:46](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=46s) [Gemma 4 in blue over a dotted geometric pattern, with presenters visible in the broadcast layout.](https://assets.ai.engineer/sites/aiecode2025/sha256/03ce8ede4d87d3770294add0410515ceda5b39237495f260f2af14694d02f56c.webp)

Gemma 4

AIE
Google DeepMind

[1:27](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=87s) [Gemma 4 title over a dotted geometric design.](https://assets.ai.engineer/sites/aiecode2025/sha256/5de95c4344e144ce674705e0db00a49a9c9b0828ab6975e350de2ac0a92cda7b.webp)

Gemma 4

AIE
AI Engineer
EUROPE

### Effective 2B and Effective 4B

Gemma’s E2B and E4B target mobile and smaller devices. The speaker says E2B uses about as much memory as a 2B model, despite having roughly 5B parameters.

[2:21](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=141s) [Effective 2B and Effective 4B beside a phone and chip illustration.](https://assets.ai.engineer/sites/aiecode2025/sha256/22bf7abbfc2b357e2e46da8f45806ae38053c790f8fd240f2ba191fe374d783a.webp)

Effective 2B
Effective 4B

Interface text:
AIE
AI Engineer
EUROPE

### 26B A4B and 31B Dense

The 26B mixture-of-experts model is described as many 4B-sized models working together. The speaker calls the 31B dense model their strongest.

[3:41](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=221s) [26B A4B and 31B Dense above a computer icon and blue particle graphics.](https://assets.ai.engineer/sites/aiecode2025/sha256/1bd3643a02f1cbfdde28d4ac38014d58241ef280e7bb25a0c68f370518b21a14.webp)

26B A4B
31B Dense

AIE
AI Engineer
EUROPE

[4:06](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=246s) [Model sizes above a desktop icon surrounded by blue particles.](https://assets.ai.engineer/sites/aiecode2025/sha256/1cad150f3cc6c8f0907728fa37615451b32f466a03ebaa113eec27c1f00de3c0.webp)

26B A4B
31B Dense

AIE
AI Engineer
EUROPE

### Gemma 4 Arena Elo scores

Gemma 4’s 31B and 26B models score 1452 and 1441, alongside models with 397B–1100B parameters that score 1425–1456.

[4:20](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=260s) [Bar chart comparing Arena Elo scores and parameter counts for Gemma 4 and other models.](https://assets.ai.engineer/sites/aiecode2025/sha256/6dd6ee269fcf3a6880750f07857267dfd00273180bcc5c2c0d8eabd3754febd8.webp)

Gemma 4
Arena Elo Score

Gemma 4 31B Thinking: 1452
Gemma 4 26B A4B Thinking: 1441
Glm 5: 1456
Kimi k2.5 Thinking: 1454
Qwen 3.5 397B A17B: 1450
Deepseek v3.2 Exp Thinking: 1425

Parameters in Billions:
Gemma 4 31B Thinking: 31B
Gemma 4 26B A4B Thinking: 26B
Glm 5: 754B
Kimi k2.5 Thinking: 1100B
Qwen 3.5 397B A17B: 397B
Deepseek v3.2 Exp Thinking: 685B

Interface text:
AIE
AI Engineer

### 26B A4B and 31B Dense in the cloud

The speaker recommends both models for cloud endpoints or desktops, arguing that their lower hardware needs make them cheaper to run for everyday tasks.

[5:02](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=302s) [Model sizes beside a cloud icon surrounded by a large cloud-shaped particle outline.](https://assets.ai.engineer/sites/aiecode2025/sha256/bc2663bea4b89bd81d7e19ba5cb35e49c8eb90ef4f983d6c1690a653f1270695.webp)

26B A4B
31B Dense

Interface text:
AIE
Engineering the future of AI

### Gemma 4 on ai.dev

The ai.dev playground highlights Gemma 4 31B IT. The speaker says Gemma models are free to try there, though he could not show the planned demo.

[5:57](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=357s) [AI playground screenshot with the Gemma 4 31B model highlighted in the settings panel.](https://assets.ai.engineer/sites/aiecode2025/sha256/73e1cb3601178f1859d43881fd8cb9856859d84a209223dbd437c050b21aab36.webp)

ai.dev

Explore Google models

Featured
Test out our most advanced and newest models.

Code and Chat
Build chatbots, agents, and code with Gemini 3.

Image Generation
Create and edit images with Nano Banana and Imagen.

Video Generation
Generate videos with Veo models, our state of the art video generation models.

Speech and Music
Explore our text to speech and music generation models.

Real-time
Real-time voice and video with Gemini Live.

Start building →

Gemma 4 31B IT
[illegible]
Google DeepMind's flagship open-weight dense model, purpose-built for maximum quality in data center environments with a 256K context window and advanced [illegible]...

Interface text:
Playground
Run settings
Get code
System instructions
Optional tone and style instructions for the model
No API Key
Switch to a paid API key to unlock higher quota and more features.
Temperature: 1
Media resolution: Default
Thinking level: High
Tools
Structured outputs
Code execution
Function calling
Start typing a prompt, use option + enter to append
Tools
Grounding with Google Search
AIE
AI Engineer EUROPE

### Efficiency and Intelligence per Parameter

The chart places Gemma 4’s 26B and 31B models near much larger models on Elo, a preference-based score. The slide claims equivalent performance to models up to 10x their size on targeted logic tasks.

[6:40](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=400s) [Gemma 4 model sizes and parameter-efficiency claims beside a model performance versus size chart.](https://assets.ai.engineer/sites/aiecode2025/sha256/7e1077de3980ba83c081cf5316ad084d653fb0f6bfd83544872ab0dd502c05af.webp)

Efficiency & "Intelligence-per-Parameter"

Gemma 4 family:
E2B, E4B, 26B A4B, 31B.

Extreme parameter efficiency:
max("intelligence-per-parameter")

Performance equivalent to models up to 10x their size on targeted logic tasks.

Model Performance VS Size
Y-axis: Elo Score
Y-axis ticks: 1350, 1360, 1370, 1380, 1390, 1400, 1410, 1420, 1430, 1440, 1450, 1460
X-axis: Total Model Size (Billion Parameters)
X-axis ticks: 10, 20, 30, 40, 50, 100, 200, 400, 500, 600, 1000
Model labels: gemma-4-31B-thinking, gemma-4-26B-A4B-thinking, qwen3.5-27b, qwen3.5-122b-a10b, qwen3.5-397b-a17b, deepseek-v3.2-exp-thinking, mistral-large-3, glm-5, kimi-k2.5-thinking, gpt-oss-120b

AIE
Engineering the future of AI

### Gemma 4 Sovereign

The speaker defines sovereignty as owning the model, adapting it to use cases, and avoiding dependence on continued service or permission to use it.

[7:00](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=420s) [Title slide reading Gemma 4 Sovereign.](https://assets.ai.engineer/sites/aiecode2025/sha256/4b1e0aaf2c31c0087301f31d24c99ce586c12933f077cb48cacf82678da8b4f4.webp)

Gemma 4 Sovereign

Interface text:
AIE
Engineering the future of AI

### Gemma License

The speaker says Gemma’s custom license was commercial-friendly, but legal review and procurement made adoption harder for sovereign institutions.

[7:32](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=452s) [Title slide reading Gemma License.](https://assets.ai.engineer/sites/aiecode2025/sha256/d112edfbafd20d166f33fdca19886df9a8e16fee18726cda3bfee7c84ee9d184.webp)

Gemma License

AIE
Engineering the future of AI

### Apache 2.0

The speaker says Gemma 4 and future models use Apache 2.0, aiming to make legal approval easier for institutions.

[8:11](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=491s) [Title slide reading Apache 2.0.](https://assets.ai.engineer/sites/aiecode2025/sha256/d76a7632350d319c0e8f6184d0f495e88ca153871366477ff21b624ad0685f60.webp)

Apache 2.0

Interface text:
AIE
Braintrust
WorkOS
OpenAI

### Enabling sovereign AI

Ukraine uses Gemma to digitize licensing; INSAIT built a Bulgarian-first LLM with Gemma 2, while GAIA adapts Gemma 3 for Brazilian Portuguese.

[8:26](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=506s) [Gemma sovereign AI examples from Ukraine, Bulgaria and Brazil.](https://assets.ai.engineer/sites/aiecode2025/sha256/7cb46b8870598f82e4a9b45030adae2664c2419343e849e82d6ee6bfbd9eebe8.webp)

Enabling sovereign AI

Gemma open models give governments and enterprises a base to build powerful sovereign AI systems, with complete control over their data and infrastructure

The Ministry of Economy, Ecology and Agriculture of Ukraine digitizes licensing process with Gemma
Learn more

INSAIT creates leading Bulgarian-first LLM with Gemma 2
Learn more

Introducing GAIA, a Brazilian Portuguese Gemma 3 model developed with ABRIA, CEIA, Nama, and Amadeus AI
Learn more

Interface text:
AIE
AI Engineer EUROPE

### Defining the Ownership Threshold

Moving a task from a cloud API to an open model depends on accuracy, hardware, latency and cost. Latency thresholds vary by task: user-facing work may need seconds, while batch processing can allow more time.

[11:38](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=698s) [Ownership criteria include task accuracy, hardware feasibility, performance and costs, illustrated by four gauges.](https://assets.ai.engineer/sites/aiecode2025/sha256/64eac115b2856aa68a913ceb592acfc2ab47e033504ff9457e6b26f3625612e9.webp)

Defining the Ownership Threshold

The Inflection Point: Transitioning from a cloud API to a fixed-cost open model.

Feasibility Criteria: Task accuracy alignment, hardware feasibility, and acceptable performance.

Cost Realities: Balancing token costs against upfront hardware/infrastructure and running costs.

Target Accuracy
91%
Target: > 90%

p95 Latency
3.0s
Target: < 4.0s

Peak RAM Usage
2.5 GB
Target: < 4.0 GB

Avg Cost / Call
$1.40
Target: < $1.50

AIE
Engineering the future of AI

### Personal & Edge

Mobile inference uses local hardware to keep data local; desktop or single-GPU inference targets a 24GB–80GB footprint. On mobile, power use also factors into cost.

[12:35](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=755s) [Slide discusses local mobile and desktop inference and battery use, beside the Google AI Edge Gallery screen.](https://assets.ai.engineer/sites/aiecode2025/sha256/5bd3c6f48ce9b4ccc15b26162227bf176b1ad50506421d2d968c79e899223e77.webp)

Personal & Edge

Personal (NPU/Mobile): Shifting execution to local "sunk-cost" hardware to maintain local data by design.

Edge (Desktop/Single-GPU): Establishing fixed-cost reasoning on a 24GB–80GB footprint.

Battery Priority: On mobile, power utilization is part of the cost calculation and is arguably more critical than raw token generation cost.

Interface text:
Google AI Edge Gallery
Discover the power of on-device AI models from the LiteRT community, featuring the all-new Gemma 4.

Try Gemma 4 today
Gemma 4 E2B & E4B are here! Try them in AI Chat, Agent Skills, or the use cases below.

AI Chat
Chat with the latest Gemma 4 model today

Agent Skills
New
Have Gemma 4 complete agentic tasks for you

Explore other use cases

4 Models
Ask Image
Ask questions about images

4 Models
Audio Scribe
Transcribe and translate audio

AIE
Engineering the future of AI

### Agent Skills on Personal & Edge

A phone demo introduces Agent Skills running with a local model. The slide contrasts mobile hardware with a 24GB–80GB desktop or single-GPU footprint and highlights battery use as a mobile cost.

[12:42](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=762s) [The Personal & Edge slide shows the Agent Skills introduction on a phone.](https://assets.ai.engineer/sites/aiecode2025/sha256/595a94d6847734c80de99a02adf1e181f8a42ca19277f84444f5ebcce03131dc.webp)

Personal & Edge

Personal (NPU/Mobile): Shifting execution to local "sunk-cost" hardware to maintain local data by design.

Edge (Desktop/Single-GPU): Establishing fixed-cost reasoning on a 24GB–80GB footprint.

Battery Priority: On mobile, power utilization is part of the cost calculation and is arguably more critical than raw token generation cost.

Introducing Agent Skills
Use specialized, high-order reasoning by loading different skills or creating your own.
Try tapping a sample prompt below to see Agent Skills in action!

Interface text:
Agent Skills
Gemma-4-E2B-it
Interactive Map
Kitchen Adventure
Type prompt...
Skills
AIE
AI Engineer EUROPE

### Personal & Edge: Calendar request

A mobile assistant receives a request to check the day’s calendar. The slide contrasts local mobile execution with 24GB–80GB desktop or single-GPU setups, where energy use also affects cost.

[13:43](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=823s) [A slide about local and edge AI beside a mobile assistant receiving a calendar request.](https://assets.ai.engineer/sites/aiecode2025/sha256/702f7d93100360a672fe75f532d5a023383b6e0a3e31ddecf550477c11d6c820.webp)

Personal & Edge

Personal (NPU/Mobile): Shifting execution to local "sunk-cost" hardware to maintain local data by design.

Edge (Desktop/Single-GPU): Establishing fixed-cost reasoning on a 24GB–80GB footprint.

Battery Priority: On mobile, power utilization is part of the cost calculation and is arguably more critical than raw token generation cost.

Interface text:
AIE
Agent Skills
Gemma-4-E2B-it
You
awesome I'm busy making breakfast can you check my calendar for the day and show me in bullet list
Model on GPU
Type prompt...
Unmute
15
Engineering the future of AI

### Personal & Edge: Calendar function call

A mobile assistant calls a calendar script while running a model on the GPU. The slide contrasts local mobile execution with desktop-scale reasoning and highlights battery use as a cost of on-device AI.

[13:47](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=827s) [A mobile assistant calls a calendar function beside the Personal & Edge slide.](https://assets.ai.engineer/sites/aiecode2025/sha256/c427a46241d855efdf635fdcf4b61512f5af603550c030d433464b86231a37b4.webp)

Personal & Edge

Personal (NPU/Mobile): Shifting execution to local "sunk-cost" hardware to maintain local data by design.

Edge (Desktop/Single-GPU): Establishing fixed-cost reasoning on a 24GB–80GB footprint.

Battery Priority: On mobile, power utilization is part of the cost calculation and is arguably more critical than raw token generation cost.

Engineering the future of AI

Interface text:
AIE
Agent Skills
Gemma-4-E2B-it
You
awesome I'm busy making breakfast can you check my calendar for the day and show me in bullet list
Model on GPU
Calling JS script "fetch-calendar/index.html"
Type prompt...
Unmute
15

### Personal & Edge: Map request

A mobile assistant processes a request to map Westfield Valley Fair. The slide contrasts local mobile and single-GPU reasoning, with battery use adding an energy cost on mobile.

[14:07](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=847s) [A mobile assistant processes a request to show a meeting location on a map.](https://assets.ai.engineer/sites/aiecode2025/sha256/605e106d0f0241c03606c0d9df2f2c90e04b9fd013b92613aeb837d51f7c8402.webp)

Personal & Edge

Personal (NPU/Mobile): Shifting execution to local "sunk-cost" hardware to maintain local data by design.

Edge (Desktop/Single-GPU): Establishing fixed-cost reasoning on a 24GB–80GB footprint.

Battery Priority: On mobile, power utilization is part of the cost calculation and is arguably more critical than raw token generation cost.

Interface text:
AIE
Agent Skills
[illegible]
• Breakfast from 8:30 AM to 9:30 AM
• Prepare demo video for Google AI Edge Gallery app from 10:00 AM to 11:00 AM
• Lunch with Amy from 12:00 PM to 1:00 PM
• Play Tennis from 2:00 PM to 3:00 PM
• Dinner with family from 5:00 PM to 6:00 PM
You
Amy wants to meet at Westfield valley Fair can you show me where that is on the map
Model on GPU
Called JS script [illegible]
I have shown you
Type prompt...
Unmute
15
Engineering the future of AI

### Personal & Edge: Map result

A phone demo displays a map of Westfield Valley Fair alongside a comparison of local mobile execution and 24GB–80GB desktop or single-GPU reasoning. The slide also treats mobile battery use as a cost.

[14:11](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=851s) [A mobile map displays the requested location beside the Personal & Edge slide.](https://assets.ai.engineer/sites/aiecode2025/sha256/d49e75f4c5768c2d02d7edb2a0dd5048157129897766f122471a7cf72a695a15.webp)

Personal & Edge

Personal (NPU/Mobile): Shifting execution to local "sunk-cost" hardware to maintain local data by design.

Edge (Desktop/Single-GPU): Establishing fixed-cost reasoning on a 24GB–80GB footprint.

Battery Priority: On mobile, power utilization is part of the cost calculation and is arguably more critical than raw token generation cost.

Interface text:
Open in Maps
Westfield Valley Fair
Ramen Nagi
JOEY Valley Fair
LEEZ GAMES
Forest Ave
Stevens Creek Blvd
[illegible map labels]
Unmute
15
AIE
Engineering the future of AI

### Personal & Edge: Wikipedia query

A phone demo answers Wikipedia questions using a model on GPU. The slide contrasts local mobile execution with a 24GB–80GB desktop GPU footprint; the speaker also weighs immediate responses against background work while charging.

[14:39](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=879s) [A mobile assistant answers a question using information from a Wikipedia query.](https://assets.ai.engineer/sites/aiecode2025/sha256/906b514dd4774d934bb8d5132e1ff77e591953895e080b681d2cd6640e315dc7.webp)

Personal & Edge

Personal (NPU/Mobile): Shifting execution to local "sunk-cost" hardware to maintain local data by design.

Edge (Desktop/Single-GPU): Establishing fixed-cost reasoning on a 24GB–80GB footprint.

Battery Priority: On mobile, power utilization is part of the cost calculation and is arguably more critical than raw token generation cost.

Agent Skills
Called JS script "query-wikipedia/index.html"

The Wikipedia query provided information about the 98th Academy Awards ceremony, which took place on March 15, 2026, and summarized the event, including the show's details, hosts, and some of the winners.

You: which film won the best picture
Model on GPU: The film that won Best Picture at the 98th Academy Awards was "One Battle After Another".

Interface text: Gemma-4-E2B-it; 6.5 s; 804.0 ms; which film won the most; Skills; Listening...; Unmute; 15

AIE
Google DeepMind

[14:44](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=884s) [Slide comparing personal and edge deployment, hardware footprints, and mobile battery priorities, alongside a phone demo.](https://assets.ai.engineer/sites/aiecode2025/sha256/43b41829af9ce0242f94cb5fc9a13b7632e234c8e76d483e39eabcc167d31f96.webp)

Personal & Edge

Personal (NPU/Mobile): Shifting execution to local "sunk-cost" hardware to maintain local data by design.

Edge (Desktop/Single-GPU): Establishing fixed-cost reasoning on a 24GB–80GB footprint.

Battery Priority: On mobile, power utilization is part of the cost calculation and is arguably more critical than raw token generation cost.

Agent Skills
Awards ceremony, which took place on March 15, 2026, and summarized the event, including the show's details, hosts, and some of the winners.

You: which film won the best picture
Model on GPU: The film that won Best Picture at the 98th Academy Awards was "One Battle After Another".

You: which film won the most nominations
Model on GPU: The film that won the most nominations at the 98th Academy Awards was "Sinners".

Interface text: Gemma-4-E2B-it; 6.5 s; 804.0 ms; 782.0 ms; Type prompt...; Skills; Unmute; 15

AIE
Google DeepMind

## Full transcript

[0:00](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=0s) [upbeat music] Hi, everyone.

[0:16](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=16s) Uh, can you hear me? Yes, you can hear me. Hi. Sorry, sorry, one minute late. I'll try to do my best to finish earlier so my friend can do, uh, pretty cool demos for you.

[0:25](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=25s) I'm Gus. This is Ian. We are from the, uh, Google DeepMind, and I'm specifically work on the Gemma product. Do any of you know what Gemma models are? Okay, perfect.

[0:37](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=37s) Perfect. Thank you very much. Uh, so today we're going to talk a little bit about ownership and open models. Uh, and, uh, well, you know who we are. But the idea is, uh, last Thursday we released our new, uh, family of models, Gemma 4, and I'm going to talk a little bit about them.

[0:53](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=53s) Uh, there is going to be more information tomorrow in the keynote by Omar, and there's another talk by Cassidy also tomorrow that you'll go into even more details. We are going to tell a little bit of the story, but the story is a little bit bigger.

[1:06](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=66s) We'll try our best here. Uh, so why does it matter? Uh, if you ask me, I work for Google, of course if you ask me what's the best model for you to try, the easiest one, I will answer for you, Gemini.

[1:18](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=78s) Gemini is the best model we have, pretty strong, multimodal, can do all kinds of things. But then there is, uh, there's more to this story than just having the strongest model possible.

[1:31](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=91s) In some situations, you want to own the model. You want to be able to run on your own hardware. You want to customize it. You want to be able to send your proprietary data that cannot leave your infrastructure.

[1:45](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=105s) So there are many situations where even the best proprietary model will not be able to help you directly. That's when you might need an open model. That's where Gemma comes in.

[1:56](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=116s) So when you think, why does Google have two family of models? Because they complement each other. So Gemini is the m-most intelligent one, can do a lot of cool stuff, but it's hosted in Google servers.

[2:07](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=127s) You need the API to access. If you need more control and access, you need an open model. That's why we have Gemma. That's why, uh, and, and we are very proud that the quality is very, very strong.

[2:18](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=138s) We're going to go some details later. But the idea is you would be able to do a lot of cool stuff with it. Among the launches we did, we released four sizes.

[2:26](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=146s) Uh, two are target to mobile, let's say like that, or, uh, or IoT or smaller devices. It's a E2B and a E4B. Uh, these names is a little bit weird.

[2:37](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=157s) We are the only ones that use this name. Uh, and the E stands for effective. And the idea here is, uh, the model, uh, uses as much as 2B, uh, what a 2B model would use as memory, but it's larger than that.

[2:53](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=173s) The 2B is around 5B, uh, parameters. But then you say, "Oh, but where is this other 3B in memory?" The f- the fun fact is that they are not really parameters from the transformers.

[3:05](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=185s) They are, uh, like mapping, uh, tokens. So you can leave them in other memory. So what you really need on your GPU memory is the two billion or the four billion.

[3:16](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=196s) Why do we do that? So that you can run these models on a phone, on a Pixel phone or b- any phone you have there. You can run these models, and they're very strong.

[3:24](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=204s) The E2B and E4B, both of them have, uh, text, uh, vision, and audio input, and they do only text output. They can do thinking. They can do coding, function calling, all these kind of cool things.

[3:37](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=217s) These all run on your phone right now. You could download it right now, right?

[3:42](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=222s) We also have other two models, which are the larger ones. We have a 26B and a [REDACTED:generic_id]. The 26 is a mixture of experts, which means that, uh, it's as if we had many other models working together where, where each of the...

[3:56](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=236s) where each one of those are like a 4B model. Why does it matter? Because it, it has twenty-six billion parameters, but it needs a space of a four billion param to do the work.

[4:07](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=247s) And this makes, uh, makes it accessible to way more hardware, to way more people, and it's still pretty strong. But our strongest model is the [REDACTED:generic_id] Dense, which is thirty-one billion parameters model, and this is really, really strong.

[4:20](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=260s) If we look into our, uh, Elo score on LM Arena, you can see that both our models are-- They are, I guess now they are fourth and seventh at least on open source models, open models.

[4:34](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=274s) And if you compare them to maybe the top twenty, thirty, all of them are at least twice, three times larger than our models. In some cases, twenty times larger.

[4:45](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=285s) So we are talking about a disproportionate amount of intelligence per size. So our [REDACTED:generic_id] model is the one I use very regularly. It can do basically anything from coding, agentic everything, multilingual, all of that.

[4:59](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=299s) So I strongly recommend you try those. They are so strong that they are, uh, o-both of them are really good to use on your, on a, as a cloud deployed model.

[5:08](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=308s) They can run on your desktop, but if you use on your server as your endpoint to do s- uh, your work, they are pretty good. And you ask, "Oh, is this the most intelligent model?"

[5:18](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=318s) No, it isn't. I'm going-- I'm, I'm very biased, and I love them, but I know the capabilities. But the question is, do you need the most intelligent model of the planet to summarize your email, to do some more menial tasks, to help you code, to do some agentic capabilities that are s- uh, searching and interacting with docs?

[5:37](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=337s) Probably not. That's why these models are so strong, because they're cheaper. They're very strong, but they're cheaper to run. They require way less hardware. A [REDACTED:generic_id] running one GPU.

[5:48](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=348s) The competitors need two hundred gigabytes of memory, which would be maybe four or five GPUs. So you can see that the price here is really, really different.

[5:59](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=359s) One easy place for you to try these models is on ai.dev, where you can try Gemini models, Veo, all the other models. But Gem- Gemma are there, both 26 and [REDACTED:generic_id].

[6:08](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=368s) You can try right now. They are free. You can play with it. Uh, and they can do-- I was, I was going to show a demo, but I, I can't now.

[6:14](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=374s) But they can do some cool stuff, which is vision plus thinking plus code execution all at the same time, right? Uh, I'll try to post something about this later.

[6:24](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=384s) But the idea is you can play with the models pretty easily and right there, r- not now. Finish, let's finish the, the, the talk and then you play with it.

[6:32](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=392s) Uh, and as I was saying, the in-the intelligence per parameter that these models bring is pretty good. It's very, very strong. And if we, we use the Elo score for LM Arena because it's a benchmark that's a, a person's preference, right?

[6:45](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=405s) We can look into academic benchmarks. They are very, very strong. But the, how the model responds to your queries, that's very important, right? That's how your customers will see, how you will see and, and interact with it.

[6:59](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=419s) So this is why this is so important. And

[7:03](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=423s) why does all this matter? One of the reasons that we care so much is because, uh, you want to, you want to-- the user to have ownership. And more than that, we are enabling sovereignty.

[7:16](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=436s) And sovereignty me-means in terms of you own the model, and you can adapt your use cases, and you, you are not susceptible to, I don't know, loss of service or for some kind of, uh, someone saying, "No, no, you cannot use this model anymore."

[7:30](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=450s) It's all available to you. And one of the changes we made last year until Gemma 3 and others, we had our specific license, a Gemma license, which is pretty good, commercial-friendly and all.

[7:42](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=462s) But there's a problem. If you have a custom license, I don't know if you have any lawyers here. Uh, if I tell you, "Oh, we have this custom license," your lawyers will look at me with that face that, "I hate you, Gus."

[7:53](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=473s) And then they will spend like 18 month doing procurement process to understand the license and trying to change, and that never works. So it's pretty hard for sovereignty institutions to adopt this kind of thing.

[8:04](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=484s) That's why we moved to Apache 2.0 for, for Gemma 4 and going forward. That makes- [audience member sneezes] Thank you. And that makes our life mu- uh, your life much easier to, to convince your legal department, let's say like that, that, "Look, we own this model.

[8:18](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=498s) We can use." So this is pretty important, and it enables many, many, uh, sovereignty institution to use our models. We have some examples. For example, ki- for example, uh, Ukraine used Gemma to, in parts of their inf-- uh, uh, services.

[8:31](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=511s) We have a, a one version of the Gemma model that was fine-tuned for Bulgarian. It was their, uh, LLM for the country. That was based on Gemma 2. We are working to make sure they use Gemma 4 now.

[8:42](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=522s) We also have a Brazilian version that is based on Gemma 3. It was fine-tuned for Portuguese. And the challenge of these models today is that they-- if you want to fine-tune Gemma to a specific language, it's becoming very hard to do that.

[8:56](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=536s) And the problem is hard because not the f-the tooling or anything. It's because the model is pretty strong on those languages already. So any gains you try to have, you might not get there.

[9:06](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=546s) So you might spend a lot of time to get like one percent. And then maybe, I don't know if it's the best use of your, uh, time. So this is good and bad at the same time 'cause-- but it's good that you can automatically use in many language.

[9:18](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=558s) You can try right now. And if you're going to, uh, the LLM Arena for languages, in many language, they are top two, three. And look, it's a [REDACTED:generic_id] model.

[9:28](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=568s) It's very, very small, right? So this is pretty good.

[9:32](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=572s) That being said, uh, I will let my colleague continue and show some demos. [microphone feedback]

[9:38](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=578s) Thank you, Gus. So, uh, one thing that I think is really important about these models is that when you think about using open models, you think about like using proprietary models.

[9:48](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=588s) We're move-- We're seeing a shift now to more kind of agentic capabilities and the kind of tasks that we're trying to do. And with that comes a cost in tokens and token generation.

[9:57](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=597s) So one benefit of, uh, taking ownership of the models is your ability to control, or in cases where you have sunken hardware cost, uh, to be able to iterate on top of that.

[10:07](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=607s) Uh, this graph on the right-hand side is from the State of AI report that OpenRouter did, and it shows the, uh... It's a bit small for you on this diagram, but have a look at that link.

[10:16](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=616s) Uh, it shows the different types of tasks that people are doing through OpenRouter at the moment. And you'll see the, the, the one that's about here, this one here is programming is right in the middle, and this is kind-- among some of the highest tasks in terms of token generation, both input and output combined.

[10:33](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=633s) So the more we have agents work and do these kind of tasks for us that have very high token generation costs, that's when you start to get more benefit from being able to take control of that i-in itself.

[10:45](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=645s) So if, for instance, you have a laptop that is capable of doing a particular task that you need to be doing, like processing a document or analyzing some data or doing some research or, in the cases, uh, Gus talked about, doing some coding that's suitable for that, then you have a GPU that you can take advantage to

[11:02](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=662s) do some of that stuff. Now, similarly, similar to what Gus said about, you know, we don't necessarily-- We still have frontier models for doing the best possible things. I wouldn't get this model to do like a, you know, a full systems architecture and redesign of your application, right?

[11:16](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=676s) It's not kind of for that. But what it is very good at doing is following very specific instructions about doing things like refactoring, analyzing, uh, generating code in, uh, in small modular bits, and you can offload a chunk of work in that style to these kind of models to be able to do that, whether it's on a

[11:33](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=693s) single GPU or on your own personal hardware. Um, and the way that we kind of think about this is like a, a set of thresholds. Like, when do we get to the point where these models are capable of doing the task, but then they also fit on the right hardware, that they also, uh, can do it with

[11:51](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=711s) the right amount of latency, depending on the, the-- if it's a task for a user, it needs to happen in a couple seconds. If it's a task where you're doing things like batch processing, you maybe have slightly different thresholds for like what needs to be done.

[12:04](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=724s) Um, and then also what the cost of actually doing that is. So if you have a sunken cost in terms of like infrastructure that you already own or that you're prepared to, uh, outlay and then operating on that, or whether you're leasing like GPU time or something else like that.

[12:18](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=738s) So these are gonna be very specific to the task that you're trying to achieve. But what, uh, what you can do with open models is you can think very carefully about like what-- which of these tasks can I f- can I fully offload or can I fully own compared to relying just on using the best possible models

[12:33](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=753s) to do, uh, that in the cloud. And, uh, an example, so Gus talked about the different types of hardware that can run these things now. I'm just gonna run this little demo in the side at the moment.

[12:45](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=765s) So we now have models that will work directly on mobile and edge devices. Uh, this example here, um, was built by Cormac's team, um, is a set of agent skills that the model is running on a phone.

[12:58](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=778s) So, uh, I'm gonna mute the microphone for that. Um, so you can talk to the model, you can show it images, uh, you can show it the world around you, and you can prompt it and chat to it.

[13:09](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=789s) And what this one is showing is that it can look through a set of skills that it has about things on the phone. So either it can take actions on the device itself, so trigger other applications, like trigger calendar apps, trigger maps apps, or you can kind of define your own skill sets.

[13:25](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=805s) And what's different now with the Gemma 4 models than we saw for the previous generation is that it's able to reason about what actions it needs to take and reliably, uh, make those function calls defined.

[13:36](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=816s) So what this app will allow you to do is, is kind of acts as like a playground. Uh, so this is Google AI Edge Gallery, and you can find it on iOS and Android.

[13:44](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=824s) And you can experiment to see what the models of this size are actually able to do. So I think this is the two billion parameter model, but there's also the four billion parameter model, depending on, uh, the size of your hardware.

[13:56](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=836s) And when we get to desktops and single GPUs, as Gus mentioned, that's where you can use the, uh, the 26 and the [REDACTED:generic_id] models, again, on your local hardware, and I'll show you how to do that in a minute.

[14:07](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=847s) But there's one kind of key point here is, well, whereas we're not paying for, uh, the price of these agents or models within tokens, we're actually paying for them in terms of energy cost if we think about it.

[14:18](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=858s) Because now you're thinking about utilization of GPUs, you're thinking about utilization of NPUs on the hardware itself. Uh, when are you gonna do these tasks? Does the user need to get a response right now when you're taking a picture of something?

[14:30](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=870s) Or is it something that you can process offline as a background task when they plug their phone in at night? So the-- what I'm trying to say here is that the thresholds and how you think about the usage of these models kind of shifts when you come to on-device or ownership because you think more about how they're

[14:46](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=886s) being executed and why they're being executed. Um, yeah, perfect. Uh, and similarly, on the enterprise side, if you don't have a piece of hardware that can, you know, run the thirty-one billion parameter model, you can now be thinking about scaling that down.

[15:01](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=901s) So maybe if you wanted to use a three hundred plus, uh, billion parameter model before, you might have need multiple GPUs. Now you can think about using a single, uh, H100 or A100 or even in some cases like an L4.

[15:15](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=915s) And then the costs obviously related to that also kind of go down. So again, it's a calculation that you'll have to do depending on your use cases, but there's ways that you could scale, for instance, running one of these models to serve, you know, a small team or to serve a company, um, depending on, uh, what you're

[15:30](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=930s) trying to do. And the f-- and the f-- and the final point is that you also have the fine-tuning component too, which is that because these models can be customized, you can deploy your own version of it.

[15:40](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=940s) So for instance, we have a variant of Gemma models called MedGemma, which is specialized for medical use cases. So if you wanted to have something that would operate on private data that you can control yourself, you can now d-- re-- feasibly deploy this to, like, one or maybe two GPUs, uh, to run that for, I don't know,

[15:57](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=957s) like a whole hospital, for instance. So these are kind of worth considering for the enterprise case. Uh, I'm gonna jump straight to demos now. Um, I've shown you some demos on the phone.

[16:08](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=968s) I'm gonna show you a quick demo, uh, here. Who-- Quick show of hands, who's ever used, uh, a, a tool called LM Studio?

[16:17](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=977s) Okay. Uh, just under half people. So LM Studio is a way that you can play around with local models. And I have here-- I have-- This is the 26B model, so this is our faster of the, the two larger models with four billion activated parameters.

[16:32](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=992s) And I'm-- At the moment, including the context, it's probably about twenty-six gigabytes in RAM. And this is an M4 Mac. So I've got unified memory. I've got up to about forty-eight gigabytes.

[16:42](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=1002s) Uh, so I can run it on this machine. And I'm just gonna run this terminal right here. Let's give that a go. Oops. Pre-showing my demo. Let's try that again.

[16:54](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=1014s) There we go. So I'm just gonna run a little process where I'm going to do some quick-- a trick-- quick translation on my device. So what it's gonna do is I've got an orchestrator on this side here, uh, which is going to hopefully kick off my agent in a minute.

[17:12](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=1032s) Let's make sure we are loaded. Let's see what LM Studio is doing. Yeah, it's just processing at the moment. And then it's gonna farm out, uh, this translation to all of these different, uh, uh, windows, and each one of them represents a different sub-agent.

[17:26](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=1046s) So this is running on my device, uh, and it's going to basically execute all these translations in one go. So I've given it, like, the Gemma 4 announcement, and I just wanna s-- wanna translate to all these different languages.

[17:38](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=1058s) So you'll see in a second it should hopefully send it over there.

[17:43](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=1063s) Three, two, one. And hopefully we should be generating translations in a second. There we go. So, so you can imagine doing any kind of agentic task on your local machine.

[17:54](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=1074s) Uh, you could have it, like, processing files. You could have it doing, uh, additional analysis. And hopefully, what you'll see in a minute is it will be able to compile all these back, and then it will generate me a quick webpage, and then you can see the results of your translation.

[18:08](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=1088s) There you go. So there's the multilinguality of the model there as well.

[18:14](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=1094s) Thank you. Uh, [clapping] Right. Uh, so in the interest of time, uh, I just want to say that the, the main next step for exploring and trying out these models is as simple as this code on the right-hand side.

[18:32](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=1112s) Um, you can take any OpenAI-compatible interface that you've got, and you can point it at a service like Ollama or LM, uh, LM, uh, LM Studio, and you can just pick out the Gemma model, and that's all you need to change code-wise to at least, uh, try it out.

[18:47](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=1127s) So the first thing we recommend you do is to drop it into existing workflows that you have to then see what the model can handle. Like, what is it working well at?

[18:55](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=1135s) What would it need tuning for? What is kind of out of its depth in terms of, like, the complexity of the task? Um, next is to kind of bolster your evaluation suites because, you know, benchmarks are great and everything for just saying what general capabilities are.

[19:08](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=1148s) But the reality is that how good the model is depends on how well it does on your task and not anybody else's task. Um, the other thing I mentioned very briefly is thinking about how you actually serve these models in the end.

[19:20](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=1160s) So if you need to run your own GPU, you need to host it. Yes, you're in control of, like, uptan-- uptime and downtime, but then there's, like, maintenance costs and there's stuff like that.

[19:29](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=1169s) So you have to be... You have to consider that as, like, one of the factors, like the ongoing costs as well, as well as any upfront CapEx costs if you buy infrastructure or hardware to do that too.

[19:39](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=1179s) Um, on mobile devices, for instance, you have to think about, like, if I'm gonna offload stuff to a phone, like, what am I supporting? What accelerators do they have?

[19:47](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=1187s) What size RAM do they have? So the conversation becomes a little bit more complex, but then there's a whole heap of things that you can unlock, like working offline or working on users' private data that never leaves their device.

[19:58](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=1198s) Um, and finally, if you wanna scale this up to enterprise levels, you have to think then again about, like, the kind of infrastructure that you're running on and what the ongoing costs are of that as well.

[20:08](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=1208s) But it does kind of unlock that. So, uh, with that, the summary is that you can use these models in pretty much any way you can think about. Experiment what kind of tasks are possible with it.

[20:19](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=1219s) Use some of the benchmarks to kind of give you an indication of, like, what's feasible. But, uh, really, we want to hear your feedback and how you get on with these and, and how you fine-tune them and what kind of things you run into, and, uh, we wanna help you on that journey as well.

[20:33](https://www.youtube.com/watch?v=SS-A8sE7hkw&t=1233s) So with that, thank you very much. [audience applauding] [upbeat music]
