Preferences Over Benchmarks: Model Routing — Archana Kamath & Tyler Gillam, DigitalOcean
AI Engineer World's Fair 2026 · 15:53
Cloud and AI infrastructure
DigitalOcean provides cloud computing and AI infrastructure for developers, startups and small-to-medium-sized businesses. Its platform spans Droplet virtual machines, managed Kubernetes, databases, storage and application hosting. The Inference Engine lets teams run AI applications through serverless, asynchronous batch or dedicated GPU deployments, using catalog models or bringing their own. Developers can test multimodal models, evaluate outputs and track usage, latency and spending alongside their application infrastructure.
Founded in 2012, DigitalOcean began with easy-to-deploy virtual machines; Ben Uretsky was a co-founder, and Paddy Srinivasan is its current CEO. Its AI engineering extends that managed-infrastructure approach to model selection. Inference Router, available in public preview, matches conversation context to natural-language task descriptions, then ranks eligible models using cost or latency data. Cache-aware routing preserves model bindings across related requests and weighs the additional input-token cost of switching models.
DigitalOcean acquired Katanemo Labs in 2026, bringing in the team behind Plano, the open-source proxy powering Inference Router. Plano’s architecture separates intent detection from model ranking and translates between provider APIs. Teams can use DigitalOcean’s managed router or self-host Plano with their own provider keys. DigitalOcean’s inference service uses pay-as-you-go pricing, connecting model access and deployment choices with its broader cloud platform.
DigitalOcean’s supplied archive contains one recording, which offers several paths into model routing: choosing models around user needs, configuring routing policies, and assessing the evidence shown in a demo. The descriptions below reflect the recording; they do not establish current product availability, pricing, or performance.
Preferences Over Benchmarks: Model Routing, presented by Archana Kamath and Tyler Gillam, is a useful starting point for understanding why benchmark rank alone may be insufficient. The speakers frame model choice around cost, latency, task fit, prompts and tools, reliability requirements, and user preferences.
Return to the same talk for its presentation of natural-language preferences, decision-tree rules, task-specific model pools, and failover policies. The presenters describe DigitalOcean’s router as making decisions in under 200 milliseconds, adding no customer charge, requiring no application-code changes, and being open sourced with the Plano proxy. Treat these as recorded claims rather than verified statements about today’s offering.
Use the evaluation and OpenCode comparison to examine the tradeoffs demonstrated. The playground evaluation reports 90% correctness for the router versus 95% for Opus, with fewer tokens and faster responses reported for the router. Displayed coding-session costs rise from $0.08 versus $0.25 to $0.14 versus $0.44 after tests and documentation. These are results shown in the recording, not general performance guarantees; the speakers themselves call the visual output comparison a “vibe check.” Their closing discussion connects routing to an ongoing loop of evaluations, caching, and personalization.
AI Engineer World's Fair 2026 · 15:53
Affiliations reflect their AIE appearances, not necessarily current employment.