AI Engineer World's Fair 2026
Research to Reality: Bringing frontier ML research to production
Loading slides…
Read the talk
Bringing Frontier ML Research into Production
Higharc’s approach connects research documentation, a modular Python monorepo, and deliberate pull-request decomposition so specialists can turn working ML prototypes into production services.
From a talk by Vaidas Razgaitis
Before you start: Familiarity with Python APIs, microservices, and pull-request review will help you follow the architecture and workflow.
From a sketched floor plan to production software
A hand-sketched floor plan has to become structured data before software can work with it. At Higharc, computer vision parses those sketches into an internal data model. Vaidas Razgaitis, a senior research engineer on the company’s Labs team, describes a research remit that extends from spatial reasoning to agents that guide users through product experiences. Custom transformers and diffusion models for image generation round out a toolkit shaped by the multidisciplinary demands of home building.
Turning those capabilities into production features requires people with different expertise to work on the same system. Platform, infrastructure, and backend engineers know how to build robust software, but may not know the research methods behind computer vision or training language models. ML researchers can combine recent papers into novel features, but may not have been responsible for production APIs. A working prototype therefore leaves a substantial handoff problem: how can each group contribute without having to acquire the other group’s entire specialty?
The handoff is a systems and process problem. Higharc organizes it around three connected decisions: make the research understandable to incoming engineers and product managers; give it a clear destination in the codebase; and design how to decompose the prototype into production-quality contributions. The document explains what exists, the repository establishes where it belongs, and the decomposition plan connects the two.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make the prototype legible
As engineering teams grow, written designs let people align before they build. Razgaitis draws on The Pragmatic Engineer’s discussion of that practice and applies it to research through a required research prototype taxonomy document, or RPT. It is a technical design document with additions that make ML work understandable to people joining the project. Higharc writes these documents in Notion; the medium matters less than making the design explicit.
Start with the domain context. In home building, a prototype might use a parti diagram, a graph representing circulation through a home, embeddings, or a latent-space representation. Those representations cannot remain unexplained vocabulary. Razgaitis’s test is to imagine a software engineer newly hired from JPMorgan: what architectural language and data representations would that person need explained before contributing? Once that context is established, state the business goal—why solving this problem matters and what value the ML tool creates.
The rest of the document exposes the engineering boundaries:
- Type safety: Define the contract between the core-product repository and the ML repository, including how types are shared and kept in sync.
- Persistence: Identify databases and other storage requirements, and record how far the prototype has progressed. Higharc prefers researchers not to spend excessive time here; unfinished persistence work can be a useful first contribution for an incoming software engineer.
- Architecture: Describe the system’s anatomy. Is it one workflow, a chain of workflows, or a system that makes external LLM calls?
- Decomposition: Explain how the prototype will be broken apart and merged into the production codebase.
Together with domain context and business value, these boundaries let collaborators see both what the research has established and where engineering work remains.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Give research an independent service boundary
The documented prototype needs a destination that can accommodate its continued evolution. Higharc maintains an all-Python ML monorepo separate from its core-product repository. Inside it, research initiatives become isolated, decoupled microservices. Data-driven entity prediction, for example, uses a custom transformer that can evolve independently of another research initiative. Razgaitis describes an approximately one-to-one researcher-to-microservice ratio in this setup.
Web-application clients send requests to a gateway, which guards those requests and routes them to the appropriate microservice. The services share a Docker bridge network. The gateway is the described client entry point; sharing a bridge does not itself enforce gateway-only communication between containers. This arrangement separates the product-facing route into the ML system from the individual services that researchers develop.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Repeat the service structure, not the research
Inside each microservice, Higharc uses a simple layered architecture: API, business logic, and data. Clearly documented specifications also help coding agents navigate the repository and assist researchers. The services layer holds the core business logic, which may call an external foundation model or use Higharc’s own model weights pulled into the build through CI/CD. Controllers wrap that logic, and API routers expose the controllers through standalone FastAPI applications.
A small Python example makes that boundary concrete. Here, an entity-prediction implementation is supplied to the application rather than embedded in its HTTP handler. The request and response models define the API shape, the controller translates between that shape and the predictor, and the predictor remains responsible for the model-specific work:
python
from typing import Protocol
from fastapi import APIRouter, FastAPI
from pydantic import BaseModel
class PredictionRequest(BaseModel):
entity_ids: list[str]
class PredictionResponse(BaseModel):
predicted_entity_ids: list[str]
class EntityPredictor(Protocol):
def predict(self, entity_ids: list[str]) -> list[str]: ...
class PredictionController:
def __init__(self, predictor: EntityPredictor) -> None:
self.predictor = predictor
def predict(self, request: PredictionRequest) -> PredictionResponse:
result = self.predictor.predict(request.entity_ids)
return PredictionResponse(predicted_entity_ids=result)
def create_app(predictor: EntityPredictor) -> FastAPI:
controller = PredictionController(predictor)
router = APIRouter()
@router.post("/predict", response_model=PredictionResponse)
def predict(request: PredictionRequest) -> PredictionResponse:
return controller.predict(request)
app = FastAPI()
app.include_router(router)
return app
This illustrates the layering with a minimal entity-ID contract; Higharc’s actual prediction schema is not specified in the talk. The same separation allows API work and model work to proceed in distinct parts of a service.
Clients still reach these standalone applications through the gateway. Each service’s root directory contains its metadata, build instructions, Dockerfile, project dependencies, and Poetry or uv lockfiles as appropriate. Razgaitis maps the structure of three production microservices to compare their common architectural backbone. A consistent shape makes it easier to see whether a new service is developing along the same engineering conventions.
Shared tooling supports those services across the monorepo. GitHub Actions handle build and deployment automation, alongside automated tests, linting, formatting, and type checks. Researchers also use Jupyter notebooks on Modal for GPU compute, additional ML studies, and an internal CLI. These tools support the work of packaging and serving microservices; the RPT remains the handoff document, and the monorepo remains the destination.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Design the path from prototype to reviewed code
A mapped-out prototype and a suitable repository still leave a design decision: how should the prototype enter that repository? Higharc treats decomposition as an explicit design problem. For a platform-wide agent feature, the team studied which axes to split the project along and what dependency graph those pieces would form. The task is to find coherent changes that can be understood and reviewed separately, while preserving the dependencies between them.
Higharc uses Graphite stacked diffs to turn proven monolithic prototypes into smaller pull requests. The practical sequence is:
- Choose the slices and dependencies. Use the decomposition design to establish which changes build on which others.
- Represent the work as a stack of PRs. A later change can remain in development while an earlier one is under review.
- Route each slice to the right specialist. Ask subject-matter experts across the organization to review the tightly scoped changes that need their expertise.
This supports asynchronous review: the author can continue working farther up the stack while a domain specialist reviews another PR.
The decomposition plan draws directly on the taxonomy document. Once the layers, architecture, persistence needs, and type contracts are visible, they can inform the boundaries between PRs. Research legibility becomes reviewability: the initial description of the system helps determine how engineering specialists can safely change and assess it.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Locate the bottleneck before changing the process
The three focus areas also provide a diagnostic sequence. Begin with research legibility: when product staff, software engineers, and AI engineers join an initiative, can they tell where to concentrate their effort? Can they identify concrete tasks that move the prototype toward production? If not, revisit how the research is explained and handed off.
Next, inspect the receiving codebase. New code should have an obvious home, with existing templates, frameworks, and patterns that contributors can follow. Repeatedly fighting old abstractions is a different signal: the team may have outgrown the repository’s architecture. In that case, clearer documentation alone will not remove the constraints imposed by the destination.
Finally, examine decomposition and delivery. Can the team consistently estimate timelines and delivery dates for moving research into the repository? Is it clear which subject-matter experts should review and help productionize the work? Difficulty answering these questions may point upstream—to research coordination, the handoff, or the codebase that must host the result. The place where delivery stalls is not necessarily the place where the underlying problem began.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
A practical guide to organizing FastAPI applications across Python modules and composing API routers.
Explains bridge-network connectivity, container name resolution, port publishing, and user-defined networks.
A 2024 guide demonstrating how to launch a Jupyter notebook with GPU compute on Modal.
Explains how dependent branches and smaller pull requests support incremental development and focused code review.
Further reading
Gergely Orosz explains how written engineering proposals, explicit reviewers, and evolving templates help teams coordinate as they grow.
Full transcript
Loading transcript…