Amit Desai

Amit Desai is a voice-interface specialist whose work has included Alexa, Roku, and his own startups. In his accepted conference recording, he describes combining voice user-interface expertise with technical approaches to a recurring problem: conversational behavior under uncertainty. When an assistant may have misunderstood a request, its choice to act, ask for confirmation, or stop can matter as much to the user as its recognition accuracy.

Voice-interface work

The accepted recording identifies Desai as having worked in voice AI across Alexa, Roku, and his own startups. The conference page presents him with a Roku affiliation but does not establish a specific Roku job title.

Deciding whether to act, confirm, or stop

In his talk, “Act, Confirm, or Stop? Smarter behavior for AI assistants, wearables & robots”, Desai makes an assistant’s response to imperfect understanding an explicit optimization problem. Recognition produces a hypothesis and a confidence score, but the assistant must still decide what to do with them. Improving that decision can reduce frustration even when recognition accuracy remains unchanged.

Desai develops the argument through a simplified music assistant with 1,000 labeled requests and 79 percent recognition accuracy. If the assistant always acts, every misunderstanding results in a wrong song. Introducing a stop behavior lets it reject low-confidence hypotheses and ask the listener to repeat the request. That prevents some wrong actions, but it also interrupts users whose requests were understood correctly. Choosing a confidence cutoff therefore requires weighing both forms of inconvenience.

OUCH: weighing the cost of each response

Desai calls this comparison the Outcome User Cost Heuristic, or OUCH. In the example, cost represents the additional time a listener needs to reach the intended song. If a request for “Kiss” produces a Chris Brown song instead of the intended Prince song, the listener must notice the mistake, interrupt playback, and ask again. Desai assigns that outcome an illustrative cost of ten seconds, compared with four seconds when the assistant stops and requests a repetition. These values are assumptions for the demonstration, not measured or universal recovery times.

The assistant can choose a confidence threshold by minimizing the combined cost of wrong actions and stops across the labeled requests. A cutoff that sounds intuitively reasonable can still create avoidable frustration: raising it prevents more wrong songs but also rejects more correct hypotheses. In the example, optimizing the threshold improves the modeled user outcome cost while leaving interpretation accuracy fixed at 79 percent. The preferred threshold depends on both the confidence distributions and the relative burden assigned to each outcome.

Adding confirmation introduces a third behavior and two boundaries: stop at low confidence, confirm in an intermediate region, and act at high confidence. Confirmation has its own costs. Listening to a correct suggestion and affirming it is easier than rejecting an incorrect suggestion and restating the request; Desai assigns these outcomes illustrative costs of two and six seconds. Confirmation is useful only where the errors it prevents justify the additional conversational turn, and changing the assumed costs changes the preferred boundaries.

How devices and consequences change the decision

The interface also changes these costs. A television can display several channel choices for selection with a remote, potentially making confirmation easier than a fully spoken exchange. Opening the wrong channel, however, may remove the viewer from the current state and make an incorrect action more disruptive.

Desai argues that the same reasoning becomes more important when assistants send emails, make phone calls, or take physical actions. In those settings, a mistake can have consequences beyond receiving an incorrect answer. He presents fixed thresholds as a simplified teaching example and proposes learned, real-time decision models for deployed systems. His central argument is that voice-product teams can improve an assistant not only through better recognition, but also by designing its next action around the effort and consequences users face when the system is wrong.