Salman Munaf is a reliability engineer whose work spans video infrastructure, distributed systems, and the operational problems of AI. He has worked as a lead site reliability engineer at TikTok, following software engineering work at Meta, and brings that infrastructure perspective to two connected questions: how to understand language-model systems in production, and how to control agents that can change the world outside a conversation.
From infrastructure to AI reliability
Munaf earned a computer-science degree with High Distinction from LUMS, marking his graduation in a public post. His subsequent career included software engineering at Meta and reliability engineering focused on video infrastructure and distributed systems at TikTok. That progression connects building software with keeping interconnected services dependable when requests fail, information becomes stale, or parts of a workflow succeed independently of others.
His work on LLM observability and GPU inference extends this operational focus into language-model infrastructure. Variable model outputs complicate the task of understanding application behavior, while inference introduces its own performance and resource concerns: token throughput, latency, GPU utilization, memory pressure, and energy efficiency. These dimensions matter together. Understanding what an AI system produces requires attention to the infrastructure delivering those outputs, including how quickly it responds and how efficiently it uses compute.
Agents add another layer of responsibility. Once a model can issue a refund, update a record, or send a customer message, an incorrect answer can become an external state change. Munaf treats the agent as a probabilistic coordinator: it chooses actions within a workflow, but its choices vary in ways that predefined service logic does not. His approach to agent reliability places deterministic controls around those decisions, with explicit limits, authoritative state, and recovery paths.
The mechanisms that make agents dependable
Resolve uncertain outcomes before repeating actions. A refund service may complete a payment even though the request times out before the agent receives confirmation. The timeout leaves the outcome unknown; an immediate retry could refund the customer twice. Munaf calls for request identifiers, idempotency keys, and status lookups so repeated requests do not duplicate effects and the agent can establish what actually happened. This protection belongs in the tool or external service that executes the operation.
Make persistence bounded. An agent trying to finish a task can repeatedly call an unhealthy dependency and amplify the original failure. Munaf combines exponential backoff with circuit breakers, which stop calls when a downstream service is failing. Limits on turns, parallel calls, and spending constrain both operational load and cost. Stopping becomes an enforced system behavior rather than an indefinite decision left to the model.
Treat agent memory as a cache. Information becomes operational state when it influences an action. Conversation history, project files, system prompts, databases, and cached information can become stale or disagree. Munaf argues that engineers should establish which source takes precedence, preserve provenance, and invalidate remembered information when the underlying source changes. Accurately recalling an obsolete policy can still produce an incorrect action.
Keep approval specific to the action. Scoped credentials, separate read and write permissions, and tool allowlists constrain what an agent can execute. Munaf also advocates action-specific human approval tied to parameters, actor, timestamp, and expiration. Permission for a $30 refund must not become permission for a subsequent $300 refund as the agent continues its workflow.
Design recovery around effects that have already happened. A workflow might update an internal ticket, send an email, and then fail to update the CRM. Some changes can be reversed; an email already delivered requires a compensating action, such as a correction or apology. Munaf calls for persisted workflow steps and explicit compensation operations so recovery addresses the actual state of each system. Traces should connect the model and prompt with retrieved context, tool requests and responses, errors, writes, and approvals, allowing engineers to reconstruct both the decision and its consequences.
Across inference infrastructure and agent execution, Munaf’s reliability perspective asks engineers to account for behavior beyond the model itself. Stronger models can reduce mistakes, but they cannot eliminate network failures, stale information, or adversarial input. The surrounding software must make consequential actions observable, bounded, and recoverable—even when the agent makes the wrong choice.
Salman Munaf explains how tool-using agents inherit remote-call failures, stale state and partial transactions—and how deterministic controls can limit the consequences of probabilistic decisions.
An agent coordinates external systems probabilistically. Deterministic controls must constrain the actions its decisions can cause.
A timeout leaves the operation’s outcome unknown. Request identifiers and status lookups help resolve it; idempotency prevents repeated requests from creating repeated effects.
Memory that influences action needs provenance, an authoritative source and invalidation. Recovery also needs defined compensation for earlier successful steps.
Better models reduce mistakes, while tool contracts, traces and recovery paths determine what happens when a mistake or infrastructure failure still occurs.
▶ Watch ↗