AI Engineer World's Fair 2026

The Prompt Is Still a Punch Card

Loading slides…

Read the talk

The Prompt Is Still a Punch Card

Natural language expanded what computers can understand, but submitting a complete request and waiting still leaves people managing the interaction. Conversational AI needs a different protocol.

From a talk by Ted Johnson

Type, submit, wait, repeat

You type a request into a small box and wait. A cursor blinks. A progress indicator cycles through whimsical activities—hullabalooing, tomfoolering, philosophizing—while the system works. Perhaps the answer is useful. Perhaps you rephrase the request and start again. The remarkable thing is how normal this sequence now feels.

Close-up of a cursor and busy indicator beside a progress bar reading 78%.Open full source frame
“Please wait…” above a progress bar marked 78%.

Ted Johnson, co-founder of JoinIn AI, wants to make that familiarity uncomfortable. After 25 years building enterprise software, collaboration systems and AI interfaces, he experienced ChatGPT as both an extraordinary advance and a disappointment in interaction design. Why should something so capable still require people to learn an unnatural way of communicating?

His framework separates three things that are easily conflated: channel, the medium carrying intent; expression, the range and richness of meaning an interface permits; and protocol, the rules governing the exchange. Improving one does not automatically improve the others. That distinction makes it possible to see what prompting changed—and what it inherited.

0:000:16
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

0:00 · section reference included

The keyboard feels natural because we learned it

The keyboard is familiar, ubiquitous and learned. Typing takes lessons and practice, and attempts to improve it have continued for generations. Dvorak and Colemak rearrange the keys to reduce finger movement. Some enthusiasts give each thumb four, eight or ten keys rather than spending both thumbs on a space bar. Even people who love keyboards recognize that their conventional arrangement is not inevitable.

Johnson shows a historical patent drawing, which he dates to about 1860, followed by the Hansen Writing Ball: a reminder that a radically different physical arrangement was possible. The broader point does not depend on a particular layout winning. We inherited an input device shaped by old constraints, learned to accommodate it, and eventually stopped noticing the accommodation.

A channel determines what signals can travel through an interface. A keyboard, microphone, screen, punch card and prompt box each offer different possibilities.

Channel or representationWhat it can carry
TextA sequence of discrete symbols
VoiceWords, timing, pitch and hesitation
DiagramSpatial relationships visible together

These are differences in signal capacity, not guarantees of understanding. A microphone can capture hesitation without a system knowing what that hesitation means. People routinely combine channels; forcing every interaction through one medium discards options we normally use without thinking.

2:072:14
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

2:07 · section reference included

An ocean of expression inside the same box

With language models, the keyboard and Submit button remained in place, but the range of acceptable input expanded dramatically. This is progress in expression: how much of what a person means the interface lets through. The physical act of typing is unchanged; what those keystrokes can communicate is not.

Assembly offers opcodes. Shells add commands, inputs and flags. Programming languages provide primitives that can be composed into increasingly sophisticated instructions. Each remains a machine-defined vocabulary within which the user must formulate intent. Natural language opens that vocabulary to ordinary requests. The slide places this progression alongside form fields, ending with open language.

Five panels labeled Opcodes, Shell, Primitives, Fields, and Open Language show code, form controls, and a natural-language request.Open full source frame
From Opcodes to Open Language: five forms of computer input.

An ordinary request contains context, nuance and intention that people often do not spell out to one another. Language models make much more of that meaning available to software. Yet, in Johnson’s framing, channels that have persisted for decades—and in some cases much longer—now carry this enormous expressive range through essentially the same narrow interaction. More meaning fits through the box, but the user still has to decide how and when to package it.

4:204:30
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

4:20 · section reference included

Faster batch is still batch

The remaining constraint is protocol: the shape of the interaction and the rules participants must follow. Johnson describes the recent LLM advance as an explosion in expression, while prompting retains the structure of punch-card batch processing.

The historical workflow made the separation explicit:

  1. Encode the entire request away from the computer.
  2. Carry the card deck to an operator and submit the job.
  3. Wait for the run to finish.
  4. Inspect the printout, find the mistake, repair the deck and resubmit.

The machine engaged with the finished package, not with the person while they were working out what they wanted.

StagePunch-card jobPrompt interaction
PrepareAssemble the deckCompose the request
SubmitDeliver the jobSend the message
WaitAwait the printoutAwait the response
RepairCorrect and resubmitRephrase and resend

Updates and summaries improve visibility, and the wait may shrink from hours or overnight to seconds or minutes. But response speed does not determine when the system is allowed to participate. If participation begins only after the user submits a complete turn, the central batch constraint remains. A voice interface that transcribes speech into a box and submits it preserves that constraint too; changing the input channel alone does not change the protocol.

6:146:26
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

6:14 · section reference included

Expertise in assembling the request

Prompt engineering often teaches people how to accommodate this protocol: ask for step-by-step thinking, provide examples, request an expert persona—or avoid one. Paste more context, or less. Communicate through Markdown documents. The advice can be useful, but its contradictions also expose how much work goes into preparing a package the machine will accept.

Overlay lists six suggestions, including thinking step by step, giving examples, asking for expertise, and pasting more or less context.Open full source frame
“Packaging a good batch” lists familiar prompting advice.

That skill resembles knowing how to assemble a punch-card deck so the job succeeds. Johnson is not dismissing prompts, punch cards or command lines; each is a powerful response to particular constraints. His question is whether those constraints still justify making users finish packaging their intent before the machine can help.

A system could instead ask a follow-up, clarify a thought as it develops, or acknowledge missing information. Johnson invokes MIT professor Sherry Turkle’s description of conversation: “Conversation is the most human and humanizing thing we do.” Her original context is human relationships; here, the quotation supplies a standard for the richness of interaction that a submit-and-wait loop leaves out.

The punched-card lineage reaches back to weaving looms, where a pattern could be set in advance and then run through the machine. Johnson uses that lineage as an analogy for inherited interaction rules. The objection is not that natural language is a poor encoding. It is that the LLM is permitted to engage only after the whole pattern has been prepared.

8:048:12
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

8:04 · section reference included

The work left outside the model

As reasoning, speech, vision, memory and planning improve, the surrounding interface can still leave the human coordinating everything. The user selects relevant context, remembers which question to ask, chooses the moment, notices ambiguity and repairs the output. A capable model behind a prompt box does not automatically take responsibility for any of those tasks.

When the interaction fails, users may conclude that they were insufficiently specific or simply do not understand AI. Johnson locates the problem elsewhere: the interface asks people to operate a new kind of intelligence through an inherited protocol. The race to expose model capabilities has moved faster than the design of the interaction around them.

10:1010:26
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

10:10 · section reference included

Hearing speech is not knowing whom it addresses

Johnson’s co-founder tried a simple interaction with an unnamed frontier company’s speech-to-speech voice mode. He asked when the next Timberwolves game would take place, and it answered quickly. He then pretended Johnson had arrived and said, “Hey, Ted, come on in.” The AI responded, “Sure, I'm here. What's on your mind?”

The failure was not answering the sports question. It was treating speech intended for another person as a new message requiring an AI reply. In this example, a one-message/one-reply protocol did not adequately represent the speaker’s intended recipient. Speaker identity, addressee and the obligation to respond are separate pieces of conversational state.

Johnson also points to GPT-Realtime-2 and reports backchannels such as mm-hmm and right in voice mode. OpenAI dates the API announcement to May 7, 2026, earlier than the late-May release timing he gives; that announcement does not establish the separate Voice trial timing or those particular backchannels. The design direction he highlights is participation during an exchange, rather than merely answering after it.

11:0911:21
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

11:09 · section reference included

Yielding, listening and following the new thread

The next clips demonstrate NVIDIA PersonaPlex, an external research model rather than JoinIn AI’s system. A person says they are thinking about starting a diet. The model begins recommending simple changes, including more vegetables and fruit. Before it finishes, the person interrupts to say they have signed up for a marathon. The model yields, acknowledges the new information and switches to running preparation: regular long runs, hydration, fueling and recovery.

The key behavior is not just cancellation of audio. The model stops, gives up the floor and follows the changed conversational thread. Johnson describes this as real turn-taking: listening and speaking in real time rather than requiring an isolated, completed input before responding.

A second clip is less task-oriented. Someone invites a conversational partner to visit the city, then works through a thought about spray-painted murals and whether people commission them. Short acknowledgments land within the ongoing speech. The listener can signal attention without converting every pause into a full answer or forcing the speaker to surrender the floor.

Conversational flow and group awareness remain different problems. Backchanneling does not by itself establish who is in the room or whether a remark was addressed to the AI. Johnson introduces JoinIn’s work at that boundary: improving the protocol’s representation of human and group conversation.

12:3312:44
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

12:33 · section reference included

A meeting has goals, participants and a floor

The JoinIn demonstration opens with greetings involving Sam and Jordan, then a question about the requirement under discussion. The answer identifies expense approvals as the requirement under discussion. Johnson explains the participation mechanism as utility-driven: the system labels statements as questions, proposals or answers, creates goals to fulfill, and takes a turn only when nobody else is speaking or holding the floor. Silence and an available turn are not necessarily the same thing.

The group wants users to approve requests faster, but the scope is initially loose. A participant asks what kinds of requests they mean. Expense approvals come first, with access requests later. Then someone tells the AI to hold while the people clarify whether they are designing expense approvals specifically or a general approval workflow. They settle on expense approvals for the first release and access requests as future scope; a participant notes that this distinction changes the data model.

A participant next asks the AI to pull the material up for everyone. This request explicitly addresses the assistant, making the recipient easy to identify. Johnson flags recipient tracking as essential because later requests will not include an explicit AI address. The system must follow both the topic and the relationships among utterances.

14:1014:17
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

14:10 · section reference included

From discussion to a captured requirement

The participants work through an approval constraint: expenses over $5,000 need a second approver, involving the manager and finance. A large expense therefore cannot be handled as a single tap; below the limit, they accept one-tap approval. The group explicitly agrees on expense approvals with a $5,000 threshold. Johnson identifies this as the moment the AI resolves its scope objective without anyone composing a dedicated prompt or pressing Submit.

Only afterward does someone explicitly ask the AI to capture the decision. Its spoken summary includes expense approvals for the first release, excludes access requests, and lets managers approve or reject an expense directly from a notification. It attributes routing above $5,000 to the finance controls policy. A compact representation of that captured content is:

json

{
  "firstRelease": ["expense approvals"],
  "outOfScope": ["access requests"],
  "notificationActions": ["approve", "reject"],
  "secondApproverAboveUSD": 5000
}

The distinction is between following the discussion to resolve an objective and responding to a later request to capture it. Those are separate conversational events.

A participant then asks to make the threshold 10,000 instead of 5,000, without explicitly addressing the AI. The assistant recognizes a possible revision and asks whether it should update the requirement to that threshold. The participant confirms. This establishes a confirmed change request; the clip does not establish authorization to override the finance policy or show a persisted requirement update.

A confirmed revision is not a demonstrated write

Constructed example: Object IDs, state labels and the JSON field secondApproverAboveUSD are teaching representations, not disclosed implementation fields. Scope, threshold values and confirmation come from the demo; the requirement identifier is withheld in the corrected public source.

Participant revision — unchanged
Actually, make the threshold 10,000, not 5,000.

Operation: Recognize the revision, ask whether to update the requirement, and receive the participant's confirmation.

First release

Before: Captured requirement
Expense approvals; access requests out of scope
After: After verbal confirmation · Unchanged
Expense approvals; access requests out of scope

Last captured second-approver rule

Before: Captured requirement
Expenses over $5,000
After: After verbal confirmation · Changed
Expenses over $5,000; persisted revision not shown

Confirmed change request

Before: Captured requirement
Not present
After: After verbal confirmation · Added
Set secondApproverAboveUSD to 10000; participant confirmed
The assistant infers the intended revision and seeks confirmation; the demonstrated outcome is a confirmed request.

The conversation then changes topics. Someone asks whether the room is free after the meeting. The AI begins answering, is interrupted by a clarification asking whether it is available until 3:00, and confirms that it is. The assistant handles a side question and an interruption without requiring the meeting to be repackaged as a new, carefully constructed task.

14:2415:44
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

15:44 · section reference included

Which burdens can the interface take over?

These examples suggest a broader role for AI: intelligence is also an interface technology. Better models behind prompts, agents and loops are only part of the opportunity. Reasoning, listening and inference can help remove the constraints that previously required people to adapt to a machine’s syntax, forms and timing.

The design question becomes: what are humans still doing only because machines used to be unable to do it? The answer need not be another chat window, voice mode or wall of Markdown. It may be a question, a pause, a sketch, a checklist, a quiet aside or no response at all. An intelligent interface can choose an appropriate channel and moment instead of making the user manage both.

Johnson expects removing these burdens to reduce friction and support adoption. Earlier interfaces—punch cards, typed commands, menus, phone gestures and prompts—each improved how people encode intent, while carrying some constraints into the next era. He describes the resulting work as translation, precision, context and repair taxes.

Putting those taxes down does not require making every interaction voice-based, promising magic or replacing human judgment. It requires computers to become more fluent with people. The deeper design challenge is not simply teaching users how to adopt AI: it is deciding how greater machine understanding should change the interface, so that people no longer have to reshape every thought before a machine can participate.

17:1917:33
Suggest correction

This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.

17:19 · section reference included

Resources

From the talk

  • NVIDIA PersonaPlexArticle12:33

    Research overview with interruption and backchannel examples, architecture, training details and evaluation context.

  • OpenAI's dated API launch announcement explains realtime reasoning, interruption handling and tool use.

Updates since the talk

  • Current meeting-assistant product with objective tracking, contextual answers and a desktop beta.

Transcript download
Full transcript

Loading transcript…