AI Engineer World's Fair 2026

Research to Reality: Bringing frontier ML research to production

Loading slides…

Read the talk

Bringing Frontier ML Research into Production

Higharc’s approach connects research documentation, a modular Python monorepo, and deliberate pull-request decomposition so specialists can turn working ML prototypes into production services.

From a talk by Vaidas Razgaitis

Before you start: Familiarity with Python APIs, microservices, and pull-request review will help you follow the architecture and workflow.

From a sketched floor plan to production software

A hand-sketched floor plan has to become structured data before software can work with it. At Higharc, computer vision parses those sketches into an internal data model. Vaidas Razgaitis, a senior research engineer on the company’s Labs team, describes a research remit that extends from spatial reasoning to agents that guide users through product experiences. Custom transformers and diffusion models for image generation round out a toolkit shaped by the multidisciplinary demands of home building.

Turning those capabilities into production features requires people with different expertise to work on the same system. Platform, infrastructure, and backend engineers know how to build robust software, but may not know the research methods behind computer vision or training language models. ML researchers can combine recent papers into novel features, but may not have been responsible for production APIs. A working prototype therefore leaves a substantial handoff problem: how can each group contribute without having to acquire the other group’s entire specialty?

Slide titled “Researchers ≠ production engineers” above a photograph of hands passing a relay baton.Open full source frame
Researchers ≠ production engineers: the baton pass.

The handoff is a systems and process problem. Higharc organizes it around three connected decisions: make the research understandable to incoming engineers and product managers; give it a clear destination in the codebase; and design how to decompose the prototype into production-quality contributions. The document explains what exists, the repository establishes where it belongs, and the decomposition plan connects the two.

0:010:27
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:01 · section reference included

Make the prototype legible

As engineering teams grow, written designs let people align before they build. Razgaitis draws on The Pragmatic Engineer’s discussion of that practice and applies it to research through a required research prototype taxonomy document, or RPT. It is a technical design document with additions that make ML work understandable to people joining the project. Higharc writes these documents in Notion; the medium matters less than making the design explicit.

Start with the domain context. In home building, a prototype might use a parti diagram, a graph representing circulation through a home, embeddings, or a latent-space representation. Those representations cannot remain unexplained vocabulary. Razgaitis’s test is to imagine a software engineer newly hired from JPMorgan: what architectural language and data representations would that person need explained before contributing? Once that context is established, state the business goal—why solving this problem matters and what value the ML tool creates.

The rest of the document exposes the engineering boundaries:

  • Type safety: Define the contract between the core-product repository and the ML repository, including how types are shared and kept in sync.
  • Persistence: Identify databases and other storage requirements, and record how far the prototype has progressed. Higharc prefers researchers not to spend excessive time here; unfinished persistence work can be a useful first contribution for an incoming software engineer.
  • Architecture: Describe the system’s anatomy. Is it one workflow, a chain of workflows, or a system that makes external LLM calls?
  • Decomposition: Explain how the prototype will be broken apart and merged into the production codebase.

Together with domain context and business value, these boundaries let collaborators see both what the research has established and where engineering work remains.

Six cards labeled Domain context, Business goal, Type safety, Persistence, Architecture, and Decomposition.Open full source frame
What an RPT forces you to answer.
3:373:58
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

3:37 · section reference included

Give research an independent service boundary

The documented prototype needs a destination that can accommodate its continued evolution. Higharc maintains an all-Python ML monorepo separate from its core-product repository. Inside it, research initiatives become isolated, decoupled microservices. Data-driven entity prediction, for example, uses a custom transformer that can evolve independently of another research initiative. Razgaitis describes an approximately one-to-one researcher-to-microservice ratio in this setup.

Web-application clients send requests to a gateway, which guards those requests and routes them to the appropriate microservice. The services share a Docker bridge network. The gateway is the described client entry point; sharing a bridge does not itself enforce gateway-only communication between containers. This arrangement separates the product-facing route into the ML system from the individual services that researchers develop.

6:396:57
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:39 · section reference included

Repeat the service structure, not the research

Inside each microservice, Higharc uses a simple layered architecture: API, business logic, and data. Clearly documented specifications also help coding agents navigate the repository and assist researchers. The services layer holds the core business logic, which may call an external foundation model or use Higharc’s own model weights pulled into the build through CI/CD. Controllers wrap that logic, and API routers expose the controllers through standalone FastAPI applications.

A small Python example makes that boundary concrete. Here, an entity-prediction implementation is supplied to the application rather than embedded in its HTTP handler. The request and response models define the API shape, the controller translates between that shape and the predictor, and the predictor remains responsible for the model-specific work:

python

from typing import Protocol

from fastapi import APIRouter, FastAPI
from pydantic import BaseModel


class PredictionRequest(BaseModel):
    entity_ids: list[str]


class PredictionResponse(BaseModel):
    predicted_entity_ids: list[str]


class EntityPredictor(Protocol):
    def predict(self, entity_ids: list[str]) -> list[str]: ...


class PredictionController:
    def __init__(self, predictor: EntityPredictor) -> None:
        self.predictor = predictor

    def predict(self, request: PredictionRequest) -> PredictionResponse:
        result = self.predictor.predict(request.entity_ids)
        return PredictionResponse(predicted_entity_ids=result)


def create_app(predictor: EntityPredictor) -> FastAPI:
    controller = PredictionController(predictor)
    router = APIRouter()

    @router.post("/predict", response_model=PredictionResponse)
    def predict(request: PredictionRequest) -> PredictionResponse:
        return controller.predict(request)

    app = FastAPI()
    app.include_router(router)
    return app

This illustrates the layering with a minimal entity-ID contract; Higharc’s actual prediction schema is not specified in the talk. The same separation allows API work and model work to proceed in distinct parts of a service.

Clients still reach these standalone applications through the gateway. Each service’s root directory contains its metadata, build instructions, Dockerfile, project dependencies, and Poetry or uv lockfiles as appropriate. Razgaitis maps the structure of three production microservices to compare their common architectural backbone. A consistent shape makes it easier to see whether a new service is developing along the same engineering conventions.

Shared tooling supports those services across the monorepo. GitHub Actions handle build and deployment automation, alongside automated tests, linting, formatting, and type checks. Researchers also use Jupyter notebooks on Modal for GPU compute, additional ML studies, and an internal CLI. These tools support the work of packaging and serving microservices; the RPT remains the handoff document, and the monorepo remains the destination.

7:598:16
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

7:59 · section reference included

Design the path from prototype to reviewed code

A mapped-out prototype and a suitable repository still leave a design decision: how should the prototype enter that repository? Higharc treats decomposition as an explicit design problem. For a platform-wide agent feature, the team studied which axes to split the project along and what dependency graph those pieces would form. The task is to find coherent changes that can be understood and reviewed separately, while preserving the dependencies between them.

Higharc uses Graphite stacked diffs to turn proven monolithic prototypes into smaller pull requests. The practical sequence is:

  1. Choose the slices and dependencies. Use the decomposition design to establish which changes build on which others.
  2. Represent the work as a stack of PRs. A later change can remain in development while an earlier one is under review.
  3. Route each slice to the right specialist. Ask subject-matter experts across the organization to review the tightly scoped changes that need their expertise.

This supports asynchronous review: the author can continue working farther up the stack while a domain specialist reviews another PR.

The decomposition plan draws directly on the taxonomy document. Once the layers, architecture, persistence needs, and type contracts are visible, they can inform the boundaries between PRs. Research legibility becomes reviewability: the initial description of the system helps determine how engineering specialists can safely change and assess it.

10:4811:01
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

10:48 · section reference included

Locate the bottleneck before changing the process

The three focus areas also provide a diagnostic sequence. Begin with research legibility: when product staff, software engineers, and AI engineers join an initiative, can they tell where to concentrate their effort? Can they identify concrete tasks that move the prototype toward production? If not, revisit how the research is explained and handed off.

Next, inspect the receiving codebase. New code should have an obvious home, with existing templates, frameworks, and patterns that contributors can follow. Repeatedly fighting old abstractions is a different signal: the team may have outgrown the repository’s architecture. In that case, clearer documentation alone will not remove the constraints imposed by the destination.

Finally, examine decomposition and delivery. Can the team consistently estimate timelines and delivery dates for moving research into the repository? Is it clear which subject-matter experts should review and help productionize the work? Difficulty answering these questions may point upstream—to research coordination, the handoff, or the codebase that must host the result. The place where delivery stalls is not necessarily the place where the underlying problem began.

Slide describing a systems and process problem with three cards: Make research legible, A monorepo that can receive new research, and Decomposition as a design problem.Open full source frame
Three focus areas for bringing research into production.
12:3112:43
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

12:31 · section reference included

Resources

From the talk

Transcript download
Full transcript

Loading transcript…