Kanish Manuja

Kanish Manuja builds production LLM gateways and conversational-AI infrastructure for customer-facing applications. A principal engineer at Twilio at the time of AI Engineer World’s Fair 2026, he focuses on the engineering choices that keep those applications useful when models are slow, providers fail, or safeguards become unavailable. His work connects distributed systems with the practical demands of conversation: responsiveness, reliable instructions, retained context, and manageable cost.

From recommendation systems to conversational AI

Manuja studied computer science and engineering at the Indian Institute of Technology Kanpur, graduating in its 2015 dual-degree cohort. That year, he co-authored research on social connections in recommendation systems with Arnab Bhattacharya. Their approach incorporated friendship and trust relationships into matrix-factorization-based collaborative filtering, examining whether social relationships could improve rating predictions beyond conventional preference matching.

By August 2019, he was a software engineer at Microsoft, working on OData developer infrastructure. His writing on continuation-token pagination addressed a subtle reliability problem: offset-based pagination can repeat records when deletions change a dataset between requests. Continuation tokens preserve ordering information so the next request can resume from the preceding page’s position. He also described how developers could customize token generation and application.

His later work at Twilio brought language models into customer-service workflows. Manuja is one of eight named inventors on a patent application for asynchronously generated customer summaries, filed in 2024 and published in 2025. The proposed system retrieves customer traits and event history, generates a concise summary with a language model, and supplies that context to a human agent alongside an incoming communication request. It addresses how generated information reaches an agent within an existing service interaction; the application describes a proposed system rather than an established deployment.

A restaurant voice-agent prototype gave Manuja a concrete way to examine the tradeoffs in conversational AI. In his account of the experiment, a realtime approach improved response latency, but instruction adherence and conversational-context behavior were weaker, and cost was higher. Drawing on that experience, he recommended separate speech-to-text, language-model, and text-to-speech stages for most production use cases and advised avoiding realtime voice-native models for the time being. His recommendation extended beyond the prototype: faster responses did not, in his assessment, outweigh the difficulties of maintaining consistent behavior and context at higher cost. This was his time-bound judgment about production readiness, rather than an established verdict on every realtime voice system.

Reliability through explicit choices

Manuja treats a gateway as middleware between applications and model providers, responsible for routing, authentication, fallbacks, rate limits, and governance. His approach to gateway architecture revolves around four competing objectives: availability, latency, guardrails, and cost. When a dependency degrades, teams cannot maximize all four. Gateway designers should give applications the controls to choose an acceptable response for their particular use case.

• Per-request fallbacks. Repeatedly calling a slow, expensive model API can consume the application’s latency budget while multiplying cost. Manuja favors trying another provider when an individual request fails and temporarily removing persistently failing providers from the request path. Shared failure counters can accelerate fleet-wide failover, while instance-local counters change behavior as deployment size changes. A fallback also needs testing and capacity: superficially compatible APIs can differ in tool-call schemas, token limits, and stop reasons, and the backup must absorb traffic during a primary outage. Streaming narrows the recovery options because switching providers cannot transparently replace output already delivered to the user.

• Model-specific latency. An aggregate gateway metric mixes workloads with very different expectations: embeddings and classification can complete quickly, while reasoning requests may take much longer. Manuja recommends measuring P99 latency by model and route and setting corresponding timeouts. Without those distinctions, a request can remain technically active while the application has effectively stopped serving its user. Fixing reasoning effort per route can reduce one source of variability; issuing a delayed second request can hedge a slow response, at the expense of another model call.

• Guardrail failure policies. Safeguards are dependencies with their own outages and latency. Failing open allows a request to proceed without the unavailable check; failing closed blocks it. Manuja recommends choosing the worst outcome the use case can tolerate, then assigning guardrails time budgets and considering secondary providers, checks, or cached decisions. Placement matters too: input checks add sequential delay, concurrent checks can suit buffered structured responses, and output checks support monitoring and auditing. Concurrent checks are harder to reconcile with streaming because content may reach the user before the check finishes.

• Centralized governance. Manuja challenges the assumption that company-wide policies require every application to share one gateway deployment. Cost tracking and rate-limit management can operate through shared plugins or code across separate deployments, even when one team manages them. Within each deployment, granular API-key separation helps limit interference from noisy tenants. Bounded queues, load shedding, and traffic prioritization protect important workloads during overload and retry storms. The goal is to give teams consistent governance without making an unnecessary shared failure point part of every application’s request path.

Across his voice-agent experiments and gateway designs, Manuja pays attention to what happens when an apparently attractive technical choice meets an actual workload. Lower latency can come with weaker conversational behavior; provider redundancy can fail without tested backup capacity; a safeguard can become the service that holds up every request. His work makes those consequences explicit so application teams can choose the behavior their customers need.