AI Engineer Europe 2026

Sovereign Escape Velocity: Ownership with Open Models — Gus Martins and Ian Ballantyne, Google DeepMind

Loading slides…

Read the talk

Sovereign Escape Velocity: Ownership with Open Models

Owning a model changes where data can go, which tasks are economical, and who operates the system. Gemma 4 illustrates those choices from phone skills to local translation agents.

From a talk by Gus Martins and Ian Ballantyne

Before you start: Familiarity with model inference, GPU memory, and API clients will help with the deployment discussion and Python example.

When proprietary data cannot leave

What happens when a model needs access to proprietary data that cannot leave your infrastructure? A hosted API may offer excellent capabilities, but it cannot satisfy that constraint by intelligence alone. You need control over where inference runs, access to the model itself, and the ability to customize it.

That is the division of responsibility between Gemini and Gemma 4 in Gus Martins and Ian Ballantyne’s Google DeepMind presentation. Martins, who works on Gemma, introduces the family released the previous Thursday, with further launch detail reserved for Omar’s keynote and Cassidy’s session. He presents Gemini as Google’s strongest, easiest starting point; Gemma supplies the control that a model accessed through Google’s hosted API cannot. The two families serve complementary deployment needs.

1:311:45
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

1:31 · section reference included

What the effective sizes mean

The launch comprises four sizes. The smaller Gemma 4 E2B and Gemma 4 E4B target phones, IoT hardware, and other constrained devices. The E means effective: the name describes the effective parameter scale, rather than counting every parameter associated with the model.

Martins explains E2B as roughly a 5B-parameter model whose transformer computation has a much smaller footprint. Additional token-mapping parameters can reside outside accelerator memory. The model card describes these as per-layer embedding lookups. This is the useful distinction behind the naming: effective parameter counts are not gigabytes of memory. Actual memory requirements also depend on representation and runtime configuration; neither E2B nor E4B is a direct GPU-memory specification.

That arrangement is intended to make local execution practical on a Pixel or another phone. Both small models accept text, vision, and audio input, but produce text output. They also support thinking, coding, and function calling, so the phone deployment is not limited to a plain text chatbot.

2:182:26
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:18 · section reference included

Active computation, total weights, and useful capability

The larger options are Gemma 4 26B A4B, a mixture of experts, and Gemma 4 31B Dense. In the mixture-of-experts model, only part of the model participates in each token’s computation. Martins describes this through roughly 4B-sized working components. That figure refers to active computation, not the space needed to store the full model. The dense 31B is presented as the strongest model in the family.

Martins estimates that the two larger models stood fourth and seventh among open models on LM Arena at talk time, without assigning a rank to each. He also describes other models in the top twenty or thirty as at least two or three times larger, and sometimes twenty times larger. These are his leaderboard snapshot and size comparison, rather than a controlled task evaluation. The practical attraction is capability at a size that can fit more deployment environments; he cites coding, agentic work, and multilingual use as regular applications of the 31B model.

3:423:56
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:42 · section reference included

Enough capability for the task

Both larger models can run on a desktop or behind a server endpoint. Hosting them in a cloud environment still leaves room for ownership: the relevant question is who controls the model and its deployment, not whether the hardware sits under someone’s desk.

Slide labeled “26B A4B” and “31B Dense,” with a blue cloud icon surrounded by a dotted cloud outline.Open full source frame
26B A4B and 31B Dense models shown with a cloud graphic.

Choose for the work that needs doing. Martins explicitly concedes that these are not the most intelligent models available. Email summarization, coding assistance, document search, and interactions with documents do not necessarily require that distinction. A sufficiently capable smaller model can make those workloads accessible on less hardware.

Martins contrasts a 31B deployment on one GPU with unnamed competitors requiring 200 GB of memory and perhaps four or five GPUs. The comparison does not specify workload or configuration. The launch announcement supplies a narrower hardware condition: unquantized bfloat16 weights fit on an 80 GB H100. Fitting weights is a starting point for deployment sizing, not a measurement of serving throughput.

For an initial trial, Martins points to ai.dev, now redirecting to Google AI Studio. At the time of the talk, he describes free access to the 26B and 31B models alongside Gemini and Veo. He mentions combining vision, thinking, and code execution, although his planned demonstration of that combination is not shown.

The emphasis on LM Arena also has a specific purpose: its Elo scores reflect human preferences about responses. Academic benchmarks help characterize capabilities, while preference evaluation addresses how a model feels to query and how customers may experience its answers. Those are related but different questions.

4:595:08
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:59 · section reference included

Licensing is part of operational independence

Ownership also means being able to adapt a model and continue operating it without depending on a provider’s continued service or permission to access an endpoint. That is the operational meaning of sovereignty here: an institution can retain and run the model it has adopted.

Through Gemma 3, the family used a custom Gemma license that Martins describes as commercially friendly. But a custom license still requires interpretation. He illustrates the institutional friction with a hypothetical eighteen-month procurement review: even favorable terms can be difficult to adopt when legal teams must evaluate unfamiliar language. Gemma 4 moves to Apache 2.0, which Martins also announces as the direction for subsequent releases. That change does not retroactively change the licenses of earlier Gemma models.

The national examples span several generations. Martins mentions Gemma being used in parts of Ukraine’s services, a Bulgarian-language model based on Gemma 2, and work toward bringing that Bulgarian effort onto Gemma 4. He then describes a Brazilian adaptation of Gemma 3 for Portuguese. The Ukrainian implementation is not specified, and the language adaptations are not named in the talk.

Those adaptations raise a changing fine-tuning question. If the base model already performs strongly in a language, tooling may no longer be the main obstacle; finding a worthwhile improvement can be harder. Martins illustrates the tradeoff as substantial work for perhaps a one-percent gain, without specifying a metric. He reports top-two-or-three language rankings for the 31B model on LM Arena in many languages, but does not identify the languages. The decision is therefore whether a particular adaptation improves the institution’s tasks enough to justify the effort.

7:037:16
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:03 · section reference included

Four thresholds for offloading agent work

Ballantyne turns from model size to the amount of work agents generate. Agentic tasks consume tokens through repeated instructions, intermediate outputs, and subsequent actions. Taking ownership of inference can make that workload more controllable, especially when suitable hardware is already paid for. Citing OpenRouter’s December 2025 State of AI report, he identifies programming as one of the high-token categories when input and output are considered together. The report describes traffic through OpenRouter, rather than all AI usage.

A laptop GPU can potentially absorb document processing, data analysis, research, or suitable coding tasks. Ballantyne draws a boundary around that last category: he would not ask this model to redesign an entire application architecture. He favors specific instructions for refactoring, analysis, and generating small modular pieces of code. Those bounded jobs provide a more tractable unit of work to move onto personal hardware or a single GPU.

ThresholdQuestion to answer
CapabilityCan the model complete this specific task?
Hardware fitCan the intended machine run it?
LatencyCan it respond within the task’s time budget?
CostDoes owned or leased compute make economic sense?

These thresholds have to hold together. Ballantyne describes user-facing work as potentially needing a response within a couple of seconds, whereas batch processing can tolerate a different schedule. Already-owned infrastructure, a planned hardware purchase, and leased GPU time each produce a different calculation. The useful question is which tasks you can fully offload and operate yourself.

9:389:48
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

9:38 · section reference included

From phone input to a defined action

The mobile demonstration, credited to Cormac’s team, puts agent skills on a phone. A user can speak, provide images, show the surrounding world, or chat with the model. The model then consults the available skills: those can trigger device applications such as calendars and maps, or invoke skills defined by the developer. The mechanism connects multimodal input to a bounded set of actions the application exposes.

“Personal & Edge” slide with a phone chat showing a skill-call entry and a populated response, beside notes on local hardware, desktop GPUs, and battery use.Open full source frame
A phone agent displays a response alongside personal and edge deployment considerations.

Ballantyne identifies reasoning about which action to take, followed by reliable calls to defined functions, as an improvement over the previous generation. The playground is Google AI Edge Gallery, available on iOS and Android. He tentatively identifies the demonstration as using the smaller 2B option; a 4B option is available for hardware with sufficient capacity.

12:4512:58
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

12:45 · section reference included

Local inference changes the operating budget

Moving up to desktops and single GPUs brings the 26B and 31B models into consideration. But removing a per-token bill does not remove inference cost. GPU and NPU utilization consumes energy, so execution timing becomes a product decision. A response to a photograph may need to arrive immediately; other processing can run offline in the background while a phone charges overnight. Ownership makes scheduling and device utilization part of the application design.

For enterprise deployments, Ballantyne contrasts earlier choices involving models above 300B parameters and multiple GPUs with possible deployments on a single H100 or A100, and an L4 in some cases. Serving a team or company still requires a calculation for the actual workload: the accelerator name alone does not determine capacity or savings.

Customization adds another reason to operate the model yourself. Ballantyne introduces MedGemma as a medical model family that can work with private data under an organization’s control. He offers a hospital deployment on one or two GPUs as an illustrative possibility, not measured hospital-wide serving capacity. The enterprise opportunity combines specialization with control over where sensitive data is processed.

13:5614:07
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

13:56 · section reference included

A local orchestrator fans out translation work

The desktop demonstration uses LM Studio to run the 26B model, the faster of the two larger options, with approximately 4B activated parameters. Ballantyne reports approximately 26 GB of RAM usage, including context, on an M4 Mac with approximately 48 GB of unified memory. This is a concrete reminder that active parameter count and total runtime memory are different quantities.

The workflow starts in a terminal:

  1. An orchestrator receives the Gemma 4 announcement as the source document.
  2. It distributes translation work to sub-agents, each represented by a separate window.
  3. The sub-agents generate translations into different languages on the same local machine.

The initial wait includes processing in LM Studio before translations begin appearing. Multiple agent windows here represent separate pieces of work, not separate physical machines.

The intended final step is to collect the translations and generate a small webpage containing the results. Ballantyne describes that aggregation prospectively; the demonstration establishes translation generation but does not clearly establish the final webpage’s contents. He offers file processing and additional analysis as other tasks that could use the same local orchestration pattern.

16:0816:17
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

16:08 · section reference included

Change the endpoint, then evaluate the workflow

The first integration step is small: point an existing OpenAI-compatible client at LM Studio or Ollama, then select the locally served Gemma model. Current compatibility documentation supports this endpoint-and-model pattern for supported APIs; it does not imply complete feature equivalence. In Python, the translation task can start with a client like this, using the model identifier exposed by the local server:

python

import os
from pathlib import Path

from openai import OpenAI

client = OpenAI(
    base_url=os.environ["LOCAL_OPENAI_BASE_URL"],
    api_key=os.environ.get("LOCAL_OPENAI_API_KEY", "local"),
)

announcement = Path("announcement.txt").read_text(encoding="utf-8")
response = client.chat.completions.create(
    model=os.environ["LOCAL_GEMMA_MODEL"],
    messages=[
        {
            "role": "user",
            "content": (
                "Translate this Gemma 4 announcement into Portuguese. "
                "Preserve model names and technical identifiers. "
                "Return only the translation.\n\n" + announcement
            ),
        }
    ],
)

translation = response.choices[0].message.content
if not translation:
    raise RuntimeError("The model returned no translation text.")

Path("announcement.pt.txt").write_text(translation, encoding="utf-8")

This makes one translation request through the local endpoint. An orchestrator can assign the same source document to separate language tasks, as in the demonstration.

The next steps are empirical:

  1. Drop the model into an existing workflow. Observe which tasks it handles well.
  2. Separate tuning needs from capability limits. Some failures may be addressable through adaptation; others may reflect task complexity beyond the model.
  3. Strengthen the evaluation suite for your tasks. General benchmarks help identify candidates, but success depends on the work your application actually asks the model to perform.

The endpoint change gets the experiment started. Task evaluation determines whether the substitution is useful.

18:1418:32
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

18:14 · section reference included

The responsibilities that come with control

Self-hosting transfers uptime, downtime, maintenance, and ongoing operating costs to the owner. Buying hardware also adds upfront capital expenditure. Those obligations belong in the same decision as model quality and inference cost; they are part of what it means to control the service.

Mobile deployment adds a support matrix: which devices are supported, which accelerators they contain, and how much RAM is available. In exchange, the application can work offline or process private information that never leaves the user’s device. Enterprise scaling requires the same attention to infrastructure sizing and continuing operating costs at a larger scope.

“Best Practices & Next Steps” slide with four bullets on drop-in evaluation, custom evaluation, serving management, and enterprise scaling, plus a Python code panel.Open full source frame
Best practices cover evaluation, serving management, and enterprise scaling alongside a Python example.

Ballantyne closes by encouraging experimentation with concrete tasks, using benchmarks as signals of what may be feasible. The feedback he asks for is equally concrete: how deployments behave, what fine-tuning changes, and which problems emerge. Those observations are what turn access to an open model into a system an organization can confidently operate.

18:1419:20
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

19:20 · section reference included

Resources

From the talk

Updates since the talk

  • Gemma 4 model cardDocumentation

    Current architecture tables, parameter definitions and evaluations, including the expanded five-model family.

Transcript download
Full transcript

Loading transcript…