Ashish Kamra

Ashish Kamra works on AI performance, bringing experience in database security, enterprise storage and cloud infrastructure to the problem of making language-model applications responsive and scalable. At AI Engineer World’s Fair 2026, he was a senior engineering manager at Red Hat. His work connects reproducible inference benchmarking with the scheduling, cache management and hardware constraints that determine how a deployed system actually behaves.

From database security to enterprise infrastructure

Kamra earned a bachelor’s degree in electronics engineering from VNIT Nagpur in 2001 and worked as a systems engineer at Tata Consultancy Services from 2001 to 2004. There, he contributed to SKIMS, later called smartGOV, a workflow and document-management system for processing government files. He moved to Purdue University in 2004 to study information security and subsequently pursued doctoral research in electrical and computer engineering.

His database intrusion detection and response research used machine learning to identify unusual database activity. The system learned patterns from users’ SQL commands, detected departures from those patterns and applied policies governing the response. A suspicious request could be suspended or marked for further scrutiny rather than receiving the same treatment as every other anomaly. Kamra implemented prototype mechanisms in PostgreSQL and evaluated their performance overhead. His research with Elisa Bertino addressed database threats and access anomalies that operating-system or network intrusion detection could miss. He received his doctorate in computer engineering in 2010, with Bertino and Arif Ghafoor as his major advisers.

Internships with EMC’s Rainfinity group preceded his move into EMC’s unified-storage organization in January 2010. He later held leadership roles at Dell EMC involving enterprise and cloud storage and containerized data services. He joined Red Hat in March 2017; his Red Hat author profile describes AI performance work focused on OpenShift AI inference and training.

Making inference performance reproducible

• Repeatable configuration tuning: Kamra co-authored Red Hat’s 2025 MLPerf inference analysis, explaining how model quantization, vLLM configuration, benchmarking software and automated tuning fit together. The team’s tuning system explored configuration parameters and recorded trials for repeatable analysis. In server-scenario experiments on NVIDIA H100 GPUs, it improved performance by 5–6% over manually engineered settings. The analysis distinguished offline batch processing from interactive serving, where first-token timing, latency bounds and accuracy requirements constrain useful throughput.
• Optimization across replicas: His 2026 account of Red Hat’s distributed and multimodal inference work extended that approach across larger deployments. For GPT-OSS-120B, the team combined Bayesian configuration tuning for a single replica with improved load balancing across replicas. Tuning a worker and distributing requests are separate problems: an efficient worker can still be underused when cluster scheduling sends work elsewhere. Kamra’s account explains both levels of optimization while presenting the results as a team contribution.

Cache-aware routing and separating prefill from decoding

In their joint World’s Fair presentation, Kamra and Yuchen Fama examined how KV cache-aware routing and prefill/decode disaggregation address different sources of inference latency. Cache-aware routing sends requests toward workers that already hold reusable prompt computations, while also considering worker load. Their demonstration illustrated the benefit: a first request took roughly three seconds, while a subsequent request reusing the same system prompt took about one second. Those timings describe the demonstration, rather than a general performance guarantee.

Prefill processes a prompt and builds its attention cache; decoding generates subsequent tokens one at a time. Prefill demands substantial computation in bursts, while decoding is sensitive to memory bandwidth and streaming latency. When both phases share a GPU, a long incoming prompt can interrupt ongoing token generation. Disaggregation places the phases in independently scalable workers: the prefill worker builds the cache, and the decode worker retrieves it across the network before continuing generation.

The joint presentation treated this separation as a workload-dependent tradeoff. Long contexts, high input-to-output ratios and strict requirements for smooth token streaming can favor disaggregation. Transferring caches adds network costs, however, so shorter contexts, low concurrency or insufficient network capacity can favor aggregated serving. Effective deployment also requires adjusting the balance of prefill and decode workers as traffic changes, coordinating cache locality, model parallelism, network topology and latency targets. This extends Kamra’s benchmarking work into a practical deployment question: which configuration delivers useful performance for the workload and service requirements at hand?